Compare commits

...

404 Commits

Author SHA1 Message Date
benvin ec581b5e5f Merge pull request 'fix(ha): gate library tasks on the scan leader by default' (#11) from benvin/scan-leader-default into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #11
2026-09-13 15:04:38 +10:00
unkin-agent b77702c851 merge: resolve CoreAppHost conflict with main, drop dead redisConnectionString local
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
2026-09-13 14:48:26 +10:00
benvin 2370787471 Merge pull request 'fix(db): collapse presentation-key groups without min(uuid)' (#10) from benvin/fix-guid-aggregate-pg into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #10
2026-09-13 14:25:19 +10:00
benvin a652d718ca Merge pull request 'fix(ha): read transcode store config from the variable form deployments set' (#9) from benvin/ha-config-wiring into main
ci/woodpecker/push/ci Pipeline was canceled
Reviewed-on: #9
2026-09-13 14:24:11 +10:00
benvin 0c707c7989 Merge pull request 'fix(devices): read devices and device options through to the database' (#8) from benvin/fix-device-cache into main
ci/woodpecker/push/ci Pipeline was canceled
Reviewed-on: #8
2026-09-13 14:23:17 +10:00
unkin-agent 91655fcb0a fix(ha): gate library tasks on the scan leader by default
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Jellyfin:ScanLeader:Enabled defaults to false and nothing sets it, so
leader election never runs and every replica executes the timer-driven
library tasks concurrently - the exact behaviour the lease prevents.

- Enable gating by default when a Redis connection string is configured
- Honour an explicit Enabled setting either way
- Carry the effective decision onto the bound options the task worker reads
- Log at startup whether gating is active
- Warn when gating is enabled but no Redis connection string is configured
2026-09-13 13:19:48 +10:00
unkin-agent 6e405af1dc fix(db): collapse presentation-key groups without min(uuid)
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
PostgreSQL has no min(uuid) aggregate, so every query that picked a group
representative with MIN over the item id failed with 42883: library browse,
search, Recently Added, the by-name endpoints and Upcoming.

- Pick the representative with an anti-join on (primary version, id)
- Cover the collapse with a repository test against a real PostgreSQL
2026-09-13 13:18:11 +10:00
unkin-agent 95220e2ed6 fix(ha): read transcode store config from the variable form deployments set
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The startup configuration only reads JELLYFIN_ prefixed environment
variables, so the bare Jellyfin__TranscodeStore__* form used by the chart,
the manifests and the README is dropped and the Redis store is never
registered. Nothing logs the selected store, so the fallback is invisible.

- Read bare Jellyfin__* variables into the Jellyfin:* configuration root
- Keep an explicit JELLYFIN_ variable winning over the bare form
- Log the selected transcode session store at startup
- Ping Redis once at startup and log an unreachable store at Error
- Log endpoints only, never the connection string
2026-09-13 13:10:52 +10:00
unkin-agent 56919b9565 fix(devices): read devices and device options through to the database
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The device cache was filled once at construction, so a token minted by one
replica was unknown to every other replica already running and a token revoked
on one replica stayed valid on the others until they restarted.

- drop the eager device and device options dictionaries
- read devices and device options from the database on every query
- push the device query filters and ordering into SQL
- open the request's database context only for the api key fallback
- cover both directions against real PostgreSQL with two manager instances
2026-09-13 13:08:47 +10:00
benvin 25269263ce Merge pull request 'test(db): run the startup migration chain against PostgreSQL in CI' (#7) from benvin/ci-postgres-migration-chain into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #7
2026-09-12 22:22:12 +10:00
benvin b183bec94b Merge pull request 'fix(db): materialise the rating list before updating' (#6) from benvin/fix-rating-levels-pg into main
ci/woodpecker/push/ci Pipeline was canceled
Reviewed-on: #6
2026-09-12 22:21:30 +10:00
unkin-agent bc42ff4b28 test(db): drop a leftover test database before recreating it
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
A server handed in through JELLYFIN_TEST_POSTGRES outlives the run, so a second
run finds the databases the first one created. Also name the failures in
Jellyfin.Database.Tests.PostgreSQL the CI step steps around.
2026-09-12 21:36:09 +10:00
unkin-agent b501ba9620 test(db): run the startup migration chain against PostgreSQL in CI
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Every PostgreSQL test carries Category=RequiresDocker and the pipeline filters
that category out, so the provider production runs on was never exercised and
two upgrade-path bugs reached it in a row. Nothing covered the whole startup
sequence either - code and schema migrations interleaved the way the migration
service orders them.

- run both database stages through JellyfinMigrationService against a seeded
  library and against a fresh install
- take the server from JELLYFIN_TEST_POSTGRES when it is set, else start a
  container
- add a pipeline step that runs the PostgreSQL tests on every push and pull
  request, with the server inside the step: the kubernetes backend has no
  docker daemon, and a service container deadlocks on the workspace volume
2026-09-12 21:28:17 +10:00
unkin-agent acab85ada3 merge the rating levels fix so the chain test has it 2026-09-12 21:20:27 +10:00
unkin-agent ad834167d5 ci: probe service containers
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/probe Pipeline was canceled
2026-09-12 21:03:15 +10:00
unkin-agent f59ab11c43 fix(db): materialise the rating list before updating
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
MigrateRatingLevels issued an ExecuteUpdate while the SELECT DISTINCT reader
was still open on the same connection. SQLite tolerates that, Npgsql does not,
so the AppInitialisation stage aborted and every PostgreSQL instance
crash-looped on startup.

- read the distinct ratings into a list before the update loop
- cover the migration against a real PostgreSQL, with NULL and empty ratings
2026-09-12 21:00:18 +10:00
benvin 10fed9409d Merge pull request 'fix(db): restore the PostgreSQL upgrade path' (#4) from benvin/fix-postgres-upgrade into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #4
2026-09-12 17:55:56 +10:00
benvin f2166df463 Merge pull request 'fix(config): read an unusable encoding.xml EncoderPreset as the default' (#5) from benvin/fix-encoder-preset-fallback into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #5
2026-09-12 17:43:53 +10:00
unkin-agent 3b60289502 fix(config): read an unusable encoding.xml EncoderPreset as the default
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
An element the enum cannot parse throws, and the configuration manager
catches that and returns defaults, so one bad preset discarded every other
encoding setting.

- serialize EncoderPreset through a string surrogate in XML
- fall back to auto for an empty, unknown or out-of-range value
2026-09-12 16:26:01 +10:00
unkin-agent 2f9b8888e9 revert: move the encoding.xml EncoderPreset fix to its own branch
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
It is an unrelated subsystem and ships as a separate change.
2026-09-12 16:21:47 +10:00
unkin-agent 65af2bcbd7 fix(db): harden the PostgreSQL upgrade against odd legacy data
- accept only the two id shapes the uuid cast parses, so misplaced hyphens
  null out instead of aborting the migration
- clear an OwnerId that already is the detached placeholder, matching the
  SQLite chain, so CleanupOrphanedExtras cannot delete the item
- recreate the placeholder item before the repoint if it is missing
- compare the migrated schema against the model, constraints included
2026-09-12 16:21:42 +10:00
unkin-agent 2e3d24cc05 fix(db): restore the PostgreSQL upgrade path
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The v12.0 rebase replaced the PostgreSQL provider's initial migration instead of
adding to it, so an existing database kept the pre-12.0 schema while the code
migrations ran against it and startup aborted on a missing NormalizedUsername.

- restore 20260305010333_InitialPostgreSql as the baseline
- add 20260306000000_UpgradeToServer12Schema carrying it to the 12.0 model
- convert OwnerId and PrimaryVersionId to uuid with explicit casts, clear
  unparseable ids and repoint dangling owners at the placeholder item
- drop orphaned permissions and preferences before UserId becomes non-nullable
- add 20260524120336_AddUniqueNormalizedUsernameIndex after the code migration
  that fills the column in
- read an empty or unknown encoding.xml EncoderPreset as the default
- cover both paths against a real PostgreSQL and guard the migration ordering
2026-09-12 15:32:12 +10:00
benvin dcb6ae396c Merge pull request 'Re-apply HA fork patches on upstream v12.0' (#3) from benvin/rebase-v12 into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #3
2026-09-12 12:48:35 +10:00
unkin-agent c438ec17ed feat(helm): expose the transcode session retention window
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The chart sets the lease duration but not the retention window that keeps an
orphaned session record available for takeover.

- add ha.transcodeStore.sessionRetentionSeconds and pass it to the statefulset
2026-09-12 12:43:46 +10:00
unkin-agent f54bc76767 chore: merge main history into the v12.0 rebase
The branch is a fresh rebase onto the v12.0 tag, so it shares no history with
main and cannot be merged normally.

- record main as a parent without changing a single file
2026-09-12 10:19:36 +10:00
unkin-agent baa16b6586 fix: make transcode leases ownership-checked and cleanup-aware
Cleanup never matched a live session because the controller registered empty
manifest and segment paths, renewal was a read-modify-write that could revert a
takeover, and the takeover script compared an ISO date to a number, so it errored.

- populate the session record's manifest and segment paths from the playlist path
- renew the lease via a Lua compare-and-set on the owning pod
- store the lease expiry as unix milliseconds so the scripts can compare it
- retain the session record past its lease so an orphan can still be taken over
- test the Redis store against a real Redis, including the renew-vs-takeover race
- drop the live stream record nothing ever read back
2026-09-12 10:19:29 +10:00
unkin-agent d825f8ac81 ci: keep the package cache off the workspace volume
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The workspace volume is 9.8G and the clone plus build output already fill 5.7G,
so a shared package cache there leaves too little free space to test against.

- restore, build and test in one step with the cache on ephemeral storage
- exclude BackupServiceTests, which requires 5GiB free on the workspace
- print workspace free space before the test run
2026-09-12 01:03:59 +10:00
unkin-agent 0148f374ee ci: grant ephemeral storage and harden the package install
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
The backup service test needs 5GiB free on the workspace and the apt mirror
occasionally serves a half-synced index, so both fail the pipeline at random.

- request and limit ephemeral storage on every step
- retry apt-get update and verify fontconfig is loadable before testing
- report workspace free space before the test run
- run the full test filter again
2026-09-12 00:57:51 +10:00
unkin-agent d8c7eb5d3d ci: skip the backup service test on the pipeline workspace
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline failed
BackupService refuses to write a backup with less than 5GiB free, which the
pipeline workspace volume does not have, so the suite fails on disk size alone.

- exclude BackupServiceTests from the test filter
- keep restore, build and test as separate steps
- install fontconfig for the Skia tests before running them
2026-09-12 00:51:47 +10:00
unkin-agent 03ac3bbc00 ci: run each test suite as its own step
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
A solution-wide test run reports one exit code, so a failing suite cannot be
identified without the pipeline logs.

- run the suites carrying HA tests as separate named steps
- run the remaining upstream suites in one step after installing fontconfig
2026-09-12 00:44:47 +10:00
unkin-agent e235673f12 ci: probe system packages before building
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
A failure anywhere in the chain currently looks the same from outside, so there
is no way to tell a package mirror problem from a compile error.

- run the fontconfig install as the first step
- split restore, build and test into named steps
- point the package cache at the shared pipeline volume
2026-09-12 00:35:03 +10:00
unkin-agent b6f7732f13 ci: run the pipeline as one step again
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
Separate steps do not share the SDK package cache, so the build step could not
see what the restore step downloaded.

- restore, build and test in a single container
- install fontconfig after the build so a mirror failure cannot be read as a build failure
- raise the memory limit to 8Gi
2026-09-12 00:27:48 +10:00
unkin-agent 43b4f143eb ci: split the pipeline into restore, build and test steps
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
A single step reports every failure the same way, so a mirror, compiler or test
failure all look identical from the outside.

- split restore, build and test into separate named steps
- probe the fontconfig package install in its own step
- share the NuGet cache through the workspace and raise the memory limit
2026-09-12 00:24:26 +10:00
unkin-agent 9b8748c778 docs: document the HA layer and the v12.0 fork base
ci/woodpecker/pr/ci Pipeline failed
ci/woodpecker/push/ci Pipeline failed
The README and fork notes still described upstream and a pre-10.11 snapshot, so
there was nothing accurate to hand an operator setting this up.

- rewrite the README as an HA setup and configuration guide
- add architecture, contributing and transcoding design notes
- rewrite FORK-DIFF.md against the v12.0 base
2026-09-11 23:58:18 +10:00
unkin-agent 673d186de8 feat(deploy): add jellyfin-ha Helm chart
Running the HA build needs Redis, PostgreSQL and shared transcode storage wired
together, which is a lot of manifests to keep in sync by hand.

- add a chart deploying the server as a StatefulSet with a PodDisruptionBudget
- ship optional Redis and PostgreSQL dependencies and a runtime config ConfigMap
- template shared media, config and transcode volume claims
- add ingress and ServiceMonitor templates
2026-09-11 23:56:51 +10:00
unkin-agent 14d843897c ci: build on Woodpecker and drop GitHub Actions
The fork is hosted on Gitea, where the upstream GitHub Actions workflows never
run and only produce noise.

- add a Woodpecker pipeline restoring, building and testing the solution
- remove the upstream .github/workflows definitions
- add Dockerfile and Dockerfile.runtime building the server image with jellyfin-web 12.0
2026-09-11 23:56:51 +10:00
unkin-agent 3b416c7fb2 feat: gate periodic library tasks behind a scan-leader lease
Timer-driven library tasks fire on every replica, so a library refresh or a
database optimise runs once per pod against the same shared library.

- add IScanLeaderLease with a Redis TTL implementation and a no-op default
- skip timer-driven runs of the gated tasks on instances without the lease
- treat an unreachable Redis as holding the lease so tasks never stop running
- leave manual and API-triggered runs ungated
2026-09-11 23:56:51 +10:00
unkin-agent e3571eb2a4 feat(session): record open live streams in the session store
Live stream ownership is only tracked in process memory, so no replica can tell
which pod holds a stream open once that pod is gone.

- persist a LiveStreamSession record when the live stream mappings change
- delete the record when the stream is closed
- keep closing the stream when the store is unreachable
2026-09-11 23:56:35 +10:00
unkin-agent 6ccabd4c72 feat(hls): register transcode sessions and renew their lease
A transcode that is not in the store is invisible to the other replicas, so the
HLS entry points have to publish and hold the lease themselves.

- register the play session in the store when ffmpeg starts
- renew the lease on a background loop and delete the session when it ends
- shorten segments and bound the playlist window when resuming a stored session
- add RecoverySegmentLengthSeconds and RecoverySegmentBufferCount encoding options
2026-09-11 23:56:35 +10:00
unkin-agent c43992630e feat: skip transcode files held by an active session
Every replica runs the transcode cleanup task against the same shared transcode
directory, so one pod deletes segments another pod is still streaming.

- read the active sessions from ITranscodeSessionStore before deleting
- keep files matching an active session manifest path or segment prefix
- abort the sweep when the store cannot be reached
2026-09-11 23:56:35 +10:00
unkin-agent e23d1a23be feat: add a durable transcode session store
Transcode state lives only in the process that started ffmpeg, so a pod restart
drops every in-flight HLS stream with no way for a peer to pick it up.

- add ITranscodeSessionStore plus the TranscodeSession and LiveStreamSession records
- add RedisTranscodeSessionStore with TTL leases and an atomic Lua takeover script
- add NullTranscodeSessionStore for single-instance deployments
- pick the store from Jellyfin:TranscodeStore:RedisConnectionString at startup
2026-09-11 23:56:19 +10:00
unkin-agent c8a693ae17 feat(tools): add SQLite to PostgreSQL migration tool
Existing installs hold their whole library in jellyfin.db, so switching to the
PostgreSQL provider needs a one-shot data move.

- add Jellyfin.DbMigrator reading SQLite tables and bulk-writing them to PostgreSQL
- validate table names against the source schema before generating SQL
- upload a pre-migration copy of the SQLite file to S3 when S3_BACKUP_BUCKET is set
- emit a per-table row-count report and support a dry-run mode
2026-09-11 23:56:19 +10:00
unkin-agent e68455f0ed feat(db): add opt-in PostgreSQL database provider
Multi-replica deployments cannot share a SQLite file, so the database has to
move to a server engine before the rest of the HA work is usable.

- add Jellyfin.Database.Providers.PostgreSQL with an EF Core Npgsql provider
- register the provider and a pooled NpgsqlDataSource when DatabaseType is Jellyfin-PostgreSQL
- accept postgresql:// URIs and POSTGRES_CONNECTION_STRING alongside CustomProviderOptions
- add container-backed provider, CRUD, concurrency and migration tests
2026-09-11 23:56:04 +10:00
Jellyfin Release Bot 6c073e19dd Bump version to 12.0 2026-09-07 21:05:07 -04:00
Cody Robibero 71787c46d1 Merge pull request #17826 from joshuaboniface/fix/bump_version
Fix bump_version by removing Jellyfin.MediaEncoding.Keyframes
2026-09-07 21:03:02 -04:00
Joshua M. Boniface 35a391e99d Fix bump_version by removing Jellyfin.MediaEncoding.Keyframes
This file previously had the version in it, but it was removed in #17582
which caused this to fail. Remove it here.
2026-09-07 20:58:17 -04:00
Cody Robibero 85ebd83bdd Merge pull request #17821 from Shadowghost/more-counts
Expose optimized ItemCounts for byName items
2026-09-07 17:53:31 -04:00
Cody Robibero 882faf2b2d Merge pull request #17723 from jellyfin/renovate/bitfaster.caching-2.x
Update dependency BitFaster.Caching to 2.6.1
2026-09-07 17:44:24 -04:00
Cody Robibero d7d6cc44de Merge pull request #17824 from Raspberry-Monster/master
Fix Blu-ray multi-angle concat generation
2026-09-07 17:44:06 -04:00
Cody Robibero 42f7a6e720 Merge pull request #17820 from Shadowghost/only-download-missing-plugin-images
Only download missing plugin images
2026-09-07 17:43:57 -04:00
Cody Robibero 96f81b0c1b Merge pull request #17775 from slevin-7/fix-skia-sharpen-perf
Apply the resize sharpening kernel directly instead of via SKImageFilter
2026-09-07 17:43:48 -04:00
Cody Robibero 7d9bd1d3e4 Merge pull request #17816 from Shadowghost/version-aware-played-state
Share played state across alternate versions
2026-09-07 17:43:10 -04:00
Cody Robibero 2ffa74e7ea Merge pull request #17819 from Shadowghost/fix-tmdb-search
Fix TMDb search result ranking
2026-09-07 17:42:38 -04:00
krvi 837c34aee2 Translated using Weblate (Faroese)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/fo/
2026-09-07 20:23:12 +00:00
renovate[bot] 0fc380b4a9 Update dependency BitFaster.Caching to 2.6.1 2026-09-07 19:46:42 +00:00
Shadowghost 2644163f57 Count distinct items for byName ItemCounts and batch every kind 2026-09-07 18:39:59 +02:00
krvi dfe57e484f Translated using Weblate (Faroese)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/fo/
2026-09-07 15:15:24 +00:00
Raspberry Kan 647c4ad090 Fix Blu-ray multi-angle concat generation
Filter Blu-ray playlist clips to AngleIndex == 0 when generating concat files to prevent clips from multiple angles from being played sequentially.
2026-09-07 22:53:22 +08:00
nextlooper42 6de9d1508f Translated using Weblate (Slovak)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/sk/
2026-09-07 12:22:59 +00:00
Anastasis Marinos b2eb088e0f Translated using Weblate (Greek)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/el/
2026-09-07 12:22:59 +00:00
Shadowghost e1e319cb5c Only download missing plugin images 2026-09-07 10:43:36 +02:00
Shadowghost 59cd5f983b Expose optimized ItemCounts for byName items 2026-09-07 10:30:01 +02:00
Shadowghost 04a7568902 Fix TMDb search result ranking 2026-09-07 08:23:31 +02:00
Cody Robibero 3e7d1158da Merge pull request #17794 from Shadowghost/plugin-install-fixes
Better handle timeouts on plugin operations
2026-09-06 12:47:45 -04:00
Shadowghost fd016cfb13 Share played state across alternate versions 2026-09-06 17:20:18 +02:00
Cody Robibero f898c35b96 Merge pull request #17791 from Shadowghost/fix-code-migration
Don't dispose application singletons after running a code migration
2026-09-06 07:45:43 -04:00
Cody Robibero bbb3a48963 Merge pull request #17810 from Shadowghost/fix-tmdb-recommendations
Fix TMDb recommendations
2026-09-06 07:37:18 -04:00
Cody Robibero 34feb3c04a Merge pull request #17763 from Shadowghost/fix-refresh-queue-and-directory-cache
Fix items being lost from the refresh queue and bound the directory service caches
2026-09-06 07:36:13 -04:00
Cody Robibero 8d108d2870 Merge pull request #17806 from jellyfin/fix/dovi-color-validation
Enforce dolby vision transfer check
2026-09-06 07:35:32 -04:00
Cody Robibero 2555ab7539 Merge pull request #17805 from Shadowghost/fix-unordered-multiepisode-nfo
Fix handling of unordered multi-episode NFOs
2026-09-06 07:34:45 -04:00
Cody Robibero edb6cd11da Merge branch 'master' into fix-code-migration 2026-09-06 07:34:30 -04:00
Cody Robibero a0a42c630e Merge pull request #17786 from Shadowghost/optimize-db
Optimize database after migrations
2026-09-06 07:33:07 -04:00
Shadowghost 63553803b1 Remove the unreachable null service provider path from code migrations 2026-09-06 09:49:39 +02:00
Shadowghost 7b638a22bb Add test 2026-09-06 09:07:22 +02:00
gnattu 011eed7622 fix dovi invalid doc 2026-09-06 14:50:30 +08:00
Shadowghost 344a6dcd2c Apply review suggestions 2026-09-06 08:38:43 +02:00
Shadowghost 79a6ac6d82 Don't read episode markers in episode titles as a multi-episode range 2026-09-06 08:04:34 +02:00
Shadowghost ca90347dc2 Resolve migration routine loggers from the application container 2026-09-06 07:54:11 +02:00
Cody Robibero 66d038c403 Merge pull request #17762 from Shadowghost/fix-scan-memory-leak
Bound change batches during a scan; keep ffprobe and image saves from failing
2026-09-06 01:28:02 -04:00
Shadowghost 2ac4dd9980 Fix TMDb recommendations 2026-09-06 06:42:53 +02:00
gnattu a1910a7b2e Enforce dolby vision transfer check
Dolby vision files having unexpected transfers now also marked as invalid, and it will now get its base video range from its base layer color transfer, as not all invalid dolby vision files are HDR now.
2026-09-06 02:53:52 +08:00
Shadowghost 9221e22498 Do not resolve bundled plugins via repos 2026-09-05 20:01:09 +02:00
Shadowghost 006809f02a Fix handling of unordered multi-episode NFOs 2026-09-05 19:32:56 +02:00
Shadowghost 93345f812e FIx naming 2026-09-05 18:49:27 +02:00
Cody Robibero 7c463f5fba Merge pull request #17674 from Oggeb1/BDMV-pgs
Fix PGS subtitles for BDMV with TrueHD
2026-09-05 12:16:25 -04:00
Shadowghost 74ce774eff Revert to doing after migration and on shutdown 2026-09-05 17:33:47 +02:00
Oggeb1 9528d0e601 Fix BDMV subtitles with external audio
Co-authored-by: gnattu <gnattu@users.noreply.github.com>
2026-09-05 17:15:00 +02:00
Cody Robibero 84589380e5 Merge pull request #17746 from kfarnung/fix/vobsub-idx-subtitle-language
Probe .idx instead of .sub for external VobSub subtitle language detection
2026-09-05 11:09:46 -04:00
Cody Robibero ffd61de4f3 Merge pull request #17799 from fmarcac/fix/syncplay-stale-session-requests
Drop SyncPlay requests from sessions that left the group
2026-09-05 11:07:56 -04:00
Cody Robibero 441a05ac88 Merge pull request #17768 from Shadowghost/fix-livetv-hls-audio-codec-and-manifest-direct-play
Fix Live TV HLS playback: bogus audio encoder and unplayable direct played manifests
2026-09-05 11:00:31 -04:00
Cody Robibero 44ecc909ff Merge pull request #17796 from fmarcac/fix/syncplay-shuffle-mode-crash
Fix crash when SyncPlay shuffle mode is set to sorted twice
2026-09-05 10:58:31 -04:00
Cody Robibero 2b0ccf2a30 Merge pull request #17800 from fmarcac/fix/syncplay-active-session-counter
Fix SyncPlay active session counter leaking on rejoin
2026-09-05 10:57:55 -04:00
Cody Robibero d4a23cd438 Merge pull request #17798 from fmarcac/fix/syncplay-ping-delay-units
Fix unit mismatch in the SyncPlay resume delay floor
2026-09-05 10:55:52 -04:00
fmarcac 7ce911a401 Clamp client reported ping in SyncPlay groups 2026-09-05 15:06:37 +02:00
fmarcac e356fe9146 Return the default ping for an empty SyncPlay group 2026-09-05 14:24:49 +02:00
fmarcac 0c05d9d1a9 Apply the SyncPlay resume delay floor in the correct unit 2026-09-05 14:24:49 +02:00
fmarcac e5bfe562bc Fix crash when SyncPlay shuffle mode is set to sorted twice 2026-09-05 14:24:48 +02:00
fmarcac 51a7d5d08a Fix SyncPlay active session counter leaking on rejoin 2026-09-05 14:24:48 +02:00
fmarcac 5acb200c02 Drop SyncPlay requests from sessions that left the group 2026-09-05 14:23:37 +02:00
Shadowghost f8470630be Retire runners that are cancelled before they start 2026-09-05 09:59:12 +02:00
Shadowghost bf5fb593e4 Do not treat timeouts as cancellations 2026-09-05 09:41:07 +02:00
Shadowghost 9c259027df Cleanup 2026-09-05 08:55:12 +02:00
Shadowghost c622a16261 Don't dispose application singletons after running a code migration 2026-09-05 07:37:09 +02:00
Shadowghost 5465e0c694 Optimize the database on startup instead of before shutdown 2026-09-05 07:36:17 +02:00
Shadowghost e5cd3381ac Limit cache size again
Co-Authored-By: Cody Robibero <cody@robibe.ro>
2026-09-05 07:09:29 +02:00
Cody Robibero c80f05fad1 Merge pull request #17767 from Shadowghost/post-scna-task-logging
Add comprehensive logging for post scan tasks
2026-09-04 21:13:54 -04:00
Shadowghost 46dd7d8e99 Invalidate the singleton DirectoryService cache on filesystem changes 2026-09-04 19:28:59 +02:00
Shadowghost 0d9c9c9ecc Fix the library scheduler never retiring its runners 2026-09-04 19:23:29 +02:00
Shadowghost 0b5bbb528a Optimize database after running migrations 2026-09-04 18:57:54 +02:00
krvi f10c84465a Translated using Weblate (Faroese)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/fo/
2026-09-04 16:54:10 +00:00
AlphaBeta 610887dc2c Translated using Weblate (Thai)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/th/
2026-09-04 15:42:44 +00:00
Shadowghost 5edb369e3b Apply review suggestion 2026-09-04 07:02:08 +02:00
Shadowghost fc37151fc4 Use platform separators in the DirectoryService path tests 2026-09-04 06:57:18 +02:00
Cody Robibero c96ff18e5b Merge pull request #17757 from felixfoertsch/fix/legacy-filter-tag-query
Fix UI stalls caused by large tag and genre sets
2026-09-03 18:36:37 -04:00
krvi 3f8315df4e Translated using Weblate (Faroese)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/fo/
2026-09-03 19:05:36 +00:00
Riccardo 1ccec11b91 Translated using Weblate (Bosnian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/bs/
2026-09-03 13:37:37 +00:00
Riccardo d18d4f9cd2 Translated using Weblate (Luxembourgish)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/lb/
2026-09-03 13:37:36 +00:00
Riccardo 978552b1c5 Translated using Weblate (Irish)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/ga/
2026-09-03 13:37:35 +00:00
Riccardo f53bb0aae9 Translated using Weblate (Faroese)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/fo/
2026-09-03 13:37:35 +00:00
Riccardo 24d8290462 Translated using Weblate (Belarusian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/be/
2026-09-03 13:37:34 +00:00
Riccardo 849437991a Translated using Weblate (Latvian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/lv/
2026-09-03 13:37:34 +00:00
Riccardo 1b184b2b35 Translated using Weblate (Romanian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/ro/
2026-09-03 13:37:33 +00:00
Riccardo fb6e075e40 Translated using Weblate (Finnish)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/fi/
2026-09-03 13:37:33 +00:00
Riccardo 3682840d54 Translated using Weblate (Slovenian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/sl/
2026-09-03 13:37:32 +00:00
Riccardo a6c0fe57c5 Translated using Weblate (Norwegian Bokmål)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/nb_NO/
2026-09-03 13:37:31 +00:00
Riccardo f4d2da180d Translated using Weblate (Croatian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/hr/
2026-09-03 13:37:30 +00:00
Riccardo b561d6aa59 Translated using Weblate (Greek)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/el/
2026-09-03 13:37:29 +00:00
Shadowghost b724e57458 Reduce comments 2026-09-03 11:51:02 +02:00
Cody Robibero 6e4d98e6db Merge pull request #17764 from Shadowghost/fix-tmdb-cache-retention
Bound the TMDb response cache so a library scan cannot fill it without limit
2026-09-02 18:00:32 -04:00
Cody Robibero 4f2f781e68 Merge pull request #17756 from Shadowghost/fix-people-creation
Fix people and artist validator creation and deletion handling
2026-09-02 18:00:22 -04:00
Albert 064de2172e Apply the resize sharpening kernel directly instead of via SKImageFilter
Since the SkiaSharp 3 update the MatrixConvolution image filter used in
SkiaEncoder.ResizeImage no longer has a fast CPU path: on the software
rasterizer it takes about 4.5 seconds per megapixel-sized image, which
turns every cold image request into a multi-second operation and makes
first-time loads of a library view take minutes.

Draw the resize without the paint filter and apply the identical 3x3
kernel (same weights, clamped edges, alpha included) directly on the
resized pixels instead. This drops a cold 1000x1500 -> 663x995 poster
render from ~4.6s to well under a second; the convolution pass itself
takes ~86ms. Output is visually unchanged.
2026-09-02 21:41:36 +02:00
Felix Förtsch c3a7de54f3 avoid correlated item value name queries 2026-09-02 11:33:54 +02:00
Felix Förtsch 5adafb446f test legacy item value filter semantics 2026-09-02 11:29:24 +02:00
Felix Förtsch 3df58ab775 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-02 11:05:36 +02:00
Shadowghost 8eb4964599 Never treat a manifest container as an audio codec or a direct play target 2026-09-02 09:29:48 +02:00
Shadowghost f0297ee9ce Add comprehensive logging for post scan tasks 2026-09-02 09:26:41 +02:00
Shadowghost d73e3d964e Optimize Caches
Co-Authored-By: Cody Robibero <cody@robibe.ro>
2026-09-02 07:06:08 +02:00
Cody Robibero b3766b00d4 Merge pull request #17761 from zerafachris/fix/fallback-font-404-missing
fix: return fallback gracefully when requested fallback font is missing
2026-09-01 20:45:05 -04:00
Shadowghost 3bb8611995 Bound the TMDb response cache so a library scan cannot fill it without limit 2026-09-01 23:04:33 +02:00
Shadowghost e5dc3b8a54 Bound the directory caches a singleton would otherwise hold for the process lifetime 2026-09-01 22:04:06 +02:00
Shadowghost 0e6c52f431 Guard the refresh queue so concurrent callers cannot lose the items they queue 2026-09-01 22:03:59 +02:00
Shadowghost ccdc69e3b0 Treat an item deleted mid-save as a no-op when saving its images 2026-09-01 21:42:07 +02:00
Shadowghost c56e14d8fb Keep a media process and its exit state usable by the caller that started it 2026-09-01 21:42:06 +02:00
Shadowghost c5f8a93513 Close a change batch on its own window so a library scan cannot grow it without bound 2026-09-01 21:42:06 +02:00
Vitalijus ea89b6ebc5 Translated using Weblate (Lithuanian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/lt/
2026-09-01 19:28:24 +00:00
Vitalijus 76418ec530 Translated using Weblate (Lithuanian)
Translation: Jellyfin/Jellyfin
Translate-URL: https://translate.jellyfin.org/projects/jellyfin/jellyfin-core/lt/
2026-09-01 18:33:23 +00:00
Shadowghost 50f08d41a4 Find dead people and artists by id, not by name 2026-09-01 19:49:25 +02:00
zerafachris 20678c9a8c fix: return fallback gracefully when requested fallback font is missing
GetFallbackFont() called .First() on the font file sequence, which throws
System.InvalidOperationException when no file matched the requested name,
causing HTTP 500. Change to .FirstOrDefault() so a missing font falls
through to the existing null guard and returns HTTP 200 OK (the empty
response is intentional to avoid breaking SubtitlesOctopus).

Fixes #17683.

Prepared with AI assistance (Claude Code, Anthropic), reviewed for correctness before submission.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-01 16:25:59 +02:00
Felix Förtsch 145258865a avoid correlated legacy item value filter queries
The legacy filter queries projected ItemValue before grouping tags and genres. EF Core translated that shape into duplicated correlated aggregates, causing multi-minute requests for libraries with many distinct values and blocking unrelated SQLite-backed API calls.

Join ItemValuesMap directly to ItemValues before grouping. This preserves type and item filtering, clean-value grouping, minimum-value selection and ordering while producing one aggregate query. Add an in-memory SQLite regression test for result semantics and both SQL shapes.
2026-09-01 14:37:44 +02:00
Shadowghost 792ce4a391 Fix people validator not creating missing people 2026-09-01 07:36:53 +02:00
Kyle Farnung 39c2885fd6 Probe .idx instead of .sub for external VobSub subtitle language detection
External VobSub subtitle pairs (.idx and .sub) were only probed via the
bare .sub file. In cases where multiple languages are present, this
results in missing language metadata.

Fix by detecting the matching .idx file during media info resolution
to run ffprobe on that file and skip processing the .sub entirely.
ffprobe will automatically find the matching (same directory,
case-sensitive base) .sub file and process both.

Added regression tests covering idx/sub pairing, unpaired files,
cross-directory pairs, and language-flagged filenames.

Fixes #17745
2026-08-30 17:29:49 -07:00
Oggeb1 4207e89a7d Merge branch 'jellyfin:master' into BDMV-pgs 2026-08-29 14:02:22 +02:00
Oskar Bali 163895b99f Fix BDMV with external subtitle
GetSubtitleStreamIndexForFfmpeg treated
external and internal subtitles the same. This
made subtitles out of sync with the video.
2026-08-29 13:52:20 +02:00
Oggeb1 c15656280e Merge branch 'jellyfin:master' into BDMV-pgs 2026-08-20 10:56:35 +02:00
Oskar Bali 0b20d7a05b Fix BDMV PGS subtitles with TrueHD
Add myself CONTRIBUTORS.md
2026-08-19 12:19:32 +02:00
benvin 2e1e445e47 Merge pull request 'ci: add Woodpecker build+test pipeline, drop GitHub Actions' (#2) from benvin/woodpecker-ci into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #2
2026-08-11 21:14:33 +10:00
benvin 4920aa871a Merge pull request 'Gate periodic library-mutating tasks behind a scan-leader lease' (#1) from benvin/scan-leader-election into main
HA Build & Push to ECR / build-and-push (push) Has been cancelled
Reviewed-on: #1
2026-08-11 21:11:58 +10:00
unkinben a06b11980e ci: add Woodpecker build+test pipeline, drop GitHub Actions
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Why:
- The fork inherited GitHub Actions workflows that target the upstream's
  self-hosted GitHub runners and do not run on this Gitea/Woodpecker
  infrastructure, so source changes (such as the scan-leader lease work)
  currently land without any CI validation.

How:
- Add .woodpecker/ci.yaml running restore, build and test of Jellyfin.sln
  on pull_request and push, using the .NET 9 SDK that global.json pins.
- Filter out RequiresDocker and Integration tests, mirroring the upstream
  test selection so the suite runs without extra services.
- Set memory-heavy resource requests/limits and a dedicated
  serviceAccountName for the build+test step.
- Remove the inherited .github/workflows/ pipelines that only run on the
  upstream's GitHub Actions runners.
2026-08-11 07:23:43 +10:00
unkinben 0008bde28e Gate periodic library-mutating tasks behind a scan-leader lease
ABI Compatibility / ABI - HEAD (pull_request) Has been cancelled
ABI Compatibility / ABI - BASE (pull_request) Has been cancelled
OpenAPI / OpenAPI - HEAD (pull_request) Has been cancelled
OpenAPI / OpenAPI - BASE (pull_request) Has been cancelled
Tests / run-phase5-tests (pull_request) Has been cancelled
Tests / run-tests (pull_request) Has been cancelled
Project Automation / Project board (pull_request) Has been cancelled
Merge Conflict Labeler / Labeling (pull_request) Has been cancelled
ABI Compatibility / ABI - Difference (pull_request) Has been cancelled
OpenAPI / OpenAPI - Difference (pull_request) Has been cancelled
OpenAPI / OpenAPI - Publish Unstable Spec (pull_request) Has been cancelled
OpenAPI / OpenAPI - Publish Stable Spec (pull_request) Has been cancelled
In a multi-pod deployment every pod runs the scheduled-task timers, so
periodic library-mutating tasks (library refresh, people/chapter refresh,
audio normalization, media-segment and keyframe extraction, collection and
user-data cleanup, database optimization) fire concurrently against the shared
database and library, duplicating work and racing each other.

Add an IScanLeaderLease abstraction that elects a single scan leader via a
Redis TTL lease keyed on the pod identity, mirroring the existing transcode
lease machinery. RedisScanLeaderLease acquires or renews the lease with an
atomic Lua script and fails safe by treating the pod as leader whenever Redis
is unreachable, so scans never stall. NullScanLeaderLease preserves the
single-instance behavior when election is disabled or no Redis connection is
configured.

Gate only the timer-driven path in ScheduledTaskWorker: when election is
enabled and a task key is in the gated set, a non-leader re-arms its trigger
and skips enqueueing. Manual and API-triggered runs bypass this path and still
run on any pod. Wiring is additive and the new worker constructor parameters
are optional, so existing behavior is unchanged when election is off.

Signed-off-by: Ben Vincent <ben@unkin.net>
2026-08-10 23:51:18 +10:00
mat d4f9c12c22 fix: downgrade PostgreSQL provider to net9.0/EF Core 9.x for v10.11.7 compat
HA Build & Push to ECR / build-and-push (push) Has been cancelled
Upstream Jellyfin 10.11.7 targets net9.0. Our PostgreSQL provider was on
net10.0 with Npgsql.EntityFrameworkCore.PostgreSQL 10.0.0 which requires
EF Core 10+. Downgrade to 9.0.4 to match upstream's EF Core 9.0.11.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 22:57:39 -04:00
ZoltyMat d46539b718 docs: add architecture, contributing guide, and trim Dockerfile comment (#4)
* docs: add architecture overview, contributing guide, and GitHub discussion draft

ARCHITECTURE.md covers server layer diagram, subsystems, and runtime info.
CONTRIBUTING.md covers dev setup, build, test, and submission workflow.
GITHUB-DISCUSSION-DRAFT.md drafts the upstream discussion post for the HA fork.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: trim verbose comment in Dockerfile.runtime

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 22:56:34 -04:00
mat 13f49305da security: remove pull_request trigger from ha-build.yml
ha-build.yml runs on self-hosted k3s runners. Having pull_request as a
trigger allows any internet user to open a PR against this public repo
and execute arbitrary code on cluster nodes (GitHub does block secret
injection on fork PRs, but runner filesystem and cluster network access
remain).

Removed pull_request trigger. Build-on-push-to-main is sufficient.
CI test feedback on PRs is covered by ci-tests.yml which uses
GitHub-hosted (ubuntu-latest) runners only.
2026-03-31 22:56:34 -04:00
ZoltyMat 7d4cef51f5 feat: add Helm chart for jellyfin-ha (#3)
Adds a production-ready Helm chart under deploy/helm/jellyfin-ha/.

Motivated by a community request on Reddit:
https://www.reddit.com/r/JellyfinCommunity/comments/1rvj17f/jellyfin_ha_on_kubernetes_redisbacked_transcode/oav7mlz/

Features:
- StatefulSet with configurable replica count (default 2 for HA)
- Redis subchart (in-cluster) wired to ITranscodeSessionStore via
  Jellyfin__TranscodeStore__RedisConnectionString env var
- Supports external Redis via ha.transcodeStore.existingSecret or
  ha.transcodeStore.redisConnectionString
- Optional in-cluster PostgreSQL StatefulSet (experimental, mirrors
  existing kubernetes/apps/media/jellyfin-postgres.yaml pattern)
- RWX config + transcode PVCs (required for multi-pod session takeover)
- Per-pod cache via volumeClaimTemplates (RWO)
- Optional NFS PV+PVC for media library
- Intel QSV / VA-API GPUgit checkout -b feat/helm-chart && git add deploy/ && legit add deploy/ && git commit -m dagit commit -m featss
2026-03-31 22:56:34 -04:00
mat a9aa2d53ed docs: add FORK-DIFF.md summarising all changes vs upstream jellyfin/jellyfin 2026-03-31 22:56:34 -04:00
mat 08b87504b4 docs: add Docker Compose, k8s manifests, bare dotnet HA setup, and DigitalOcean link 2026-03-31 22:56:34 -04:00
mat df27faaa41 docs: rewrite README with HA setup guide, config reference, and architecture 2026-03-31 22:56:34 -04:00
Copilot fe9ea18918 Phase 5.2.4: Tune HLS segmentation for HA recovery (#29)
* Initial plan

* Phase 5.2.4: Tune HLS behavior for recovery with configurable segment parameters

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

* fix: SA1516 — add blank line between MakeLiveStreamKey and Clone helpers

StyleCop SA1516 requires elements to be separated by blank lines.
Missing blank line at line 370 caused build failure in Phase 5 tests.

Closes #28

* fix: SA1516 — blank line in InMemoryTranscodeSessionStore between helpers

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
Co-authored-by: mat <mstrommen@gmail.com>
2026-03-31 22:56:30 -04:00
Copilot e71efc3f97 Fix SessionManager._activeLiveStreamSessions for HA pod takeover safety (#27)
* Initial plan

* Issue 5.2.3b: Fix SessionManager._activeLiveStreamSessions for takeover safety

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

* ci: trigger CI run for PR #27 review

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
Co-authored-by: mat <mstrommen@gmail.com>
2026-03-31 22:56:30 -04:00
Copilot 9817185fa3 Add lease-aware cleanup to DeleteTranscodeFileTask (#25)
* Initial plan

* Add GetActiveSessionsAsync to ITranscodeSessionStore and update DeleteTranscodeFileTask for lease-aware cleanup

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

* Fix Redis exception propagation in GetActiveSessionsAsync for safe abort behavior

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

* fix: use KeysAsync to resolve CA1849 analyzer violation

Replace synchronous IServer.Keys() with async IServer.KeysAsync()
using await foreach to satisfy CA1849 (TreatWarningsAsErrors).

CA1849: 'IServer.Keys()' synchronously blocks.
Await 'IServer.KeysAsync()' instead.

Line 161 in RedisTranscodeSessionStore.GetActiveSessionsAsync.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
Co-authored-by: mat <mstrommen@gmail.com>
2026-03-31 22:56:30 -04:00
Copilot c2a11f3e68 Phase 5.2.2a: Register HLS sessions in ITranscodeSessionStore + lease renewal (#23)
* Initial plan

* Phase 5.2.2a: Register HLS sessions in ITranscodeSessionStore + lease renewal

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:56:30 -04:00
Copilot d60ae43b59 Wire ITranscodeSessionStore to Redis-backed impl with NullTranscodeSessionStore fallback and DI registration (#21)
* Initial plan

* feat: add Redis-backed ITranscodeSessionStore with NullTranscodeSessionStore fallback and DI registration

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

* refactor: add code review improvements - lease expiry comment, Redis connection error handling

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:56:30 -04:00
Copilot a187ab18b8 Add ITranscodeSessionStore interface and HA recovery unit tests (#19)
* Initial plan

* Add ITranscodeSessionStore interface, TranscodeSession record, InMemoryTranscodeSessionStore fake, and HA unit tests"

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:56:30 -04:00
ZoltyMat cf5268c1e4 docs: Phase 5.1.1 - HA-TRANSCODING-DESIGN.md transcode lifecycle audit (#17)
Map the exact transcode lifecycle before Phase 5.2 code changes begin.
No functional code changes.

Key findings documented:
- _activeTranscodingJobs: process-local, not persisted across pod restarts
- playSessionId: caller-supplied nullable string; three HA failure modes
- DeleteTranscodeFileTask: age-only cleanup, unsafe for shared NFS storage
- _activeLiveStreamSessions: process-local ConcurrentDictionary, cannot be
  inherited by a takeover pod without durable store rehydration
- NFSv3 confirmed (nfsvers=3). Close-to-open consistency advisory; recovery
  must skip the last incomplete segment and restart one segment earlier
- Minimum recovery state: 10 fields including server-generated sessionId,
  ownerPod, manifestPath, lastCompletedSegmentIndex, lastHeartbeatUtc

Closes Phase 5 Issue 5.1.1
2026-03-31 22:56:30 -04:00
ZoltyMat f405e17c96 ci: Phase 5.0.4b - add run-phase5-tests gate to ci-tests.yml (#16)
Add a dedicated 'run-phase5-tests' job that runs the three test assemblies
most affected by Phase 5 HLS session-sharing and PostgreSQL media-encoding
changes in parallel with the existing full-matrix run-tests job.

Targeted assemblies:
- tests/Jellyfin.Api.Tests (HLS controller surface)
- tests/Jellyfin.MediaEncoding.Hls.Tests (transcode lifecycle)
- tests/Jellyfin.Server.Implementations.Tests (RedisTranscodeSessionStore)

Each test step filters Category!=RequiresDocker to exclude Docker-dependent
tests that require Testcontainers. Coverage results are merged separately
into merged-phase5/ to avoid collisions with the full-matrix merged/ dir.

Closes #(Phase 5.0.4b)
2026-03-31 22:56:30 -04:00
ZoltyMat 67300ec17f ci: add PR trigger, copilot/* and feat/phase* branches, concurrency block (issue 5.0.0a) (#15)
- Add pull_request: trigger so Copilot agent branches get CI coverage
- Add feat/phase* and copilot/* to push branch list
- Add concurrency block to cancel stale runs on same ref
- Set push: false on PR events (build only, no ECR push for PRs)
2026-03-31 22:56:30 -04:00
mat 2b4a0a2314 fix(docker): bundle jellyfin-web into Dockerfile.runtime (the CI-used file)
Previous fixes went to Dockerfile, but CI uses Dockerfile.runtime.
Add webclient stage: installs jellyfin-web=10.11.6+deb12 from the
Jellyfin bookworm apt repo (correct version suffix: +deb12, not +1).
Web assets land at /usr/share/jellyfin/web/ and are copied to
/jellyfin/jellyfin-web/ in the runtime image.
Add --webdir flag to ENTRYPOINT explicitly.
2026-03-31 22:56:30 -04:00
mat d84a37e1d2 fix(docker): use jellyfin-web 10.11.6+deb12 from jellyfin apt repo
Previous attempt used 10.9.11+1 which does not exist — the bookworm
repo uses +deb12 suffix (e.g. 10.11.6+deb12) and only has >= 10.11.x.
This caused an empty /jellyfin/jellyfin-web/ at runtime, making the
server crash with 'content directory is either invalid or empty'.

Fix: install jellyfin-web=10.11.6+deb12 from the official Jellyfin
bookworm apt repo. Assets land at /usr/share/jellyfin/web/ and are
COPY'd to /jellyfin/jellyfin-web/ in the runtime image.
2026-03-31 22:56:30 -04:00
mat 465b6c5d9e fix(docker): install jellyfin-web 10.9.11 via apt repo into /jellyfin/jellyfin-web
Pulling from jellyfin/jellyfin:10.9.11 multi-stage yielded an empty
/jellyfin/jellyfin-web directory (the official image likely has a different
internal structure). Switch to the official Jellyfin apt repo instead:
  apt-get install jellyfin-web=10.9.11+1
  web assets land at /usr/share/jellyfin/web/
  COPY to /jellyfin/jellyfin-web/ in runtime image

--webdir /jellyfin/jellyfin-web is already in ENTRYPOINT.
2026-03-31 22:56:30 -04:00
mat b975195252 fix(docker): bundle jellyfin-web 10.9.11 from official image into wwwroot
wwwroot/ only contained api-docs (Swagger) — the web client was never bundled.
Add a webclient stage that pulls /jellyfin/jellyfin-web from jellyfin/jellyfin:10.9.11
and copies it to /jellyfin/jellyfin-web in the runtime image.

Pass --webdir /jellyfin/jellyfin-web to the entrypoint so the server serves the UI.
jellyfin-web 10.9.11 is API-compatible with the 10.12.0 server fork.
Update to a matching 10.12.x web client once that release ships upstream.
2026-03-31 22:56:30 -04:00
mat 1c28df61df fix(ha): PostgreSQL migration backup no-op + URI connection string support
- MigrationBackupFast/RestoreBackupFast/DeleteBackup return no-ops for
  PostgreSQL; pre-migration backups are handled by jellyfin-pg-backup
  CronJob, not the automated backup path that throws NotSupportedException

- ServiceCollectionExtensions: detect postgresql:// / postgres:// URI
  format in POSTGRES_CONNECTION_STRING and convert to ADO.NET key=value
  format before passing to NpgsqlDataSourceBuilder (which requires it)

Closes startup crash: 'Automated migration backups are not supported for
PostgreSQL' on first boot with a fresh database.
2026-03-31 22:56:30 -04:00
mat c69b8ccc53 fix: add missing using for IServerConfigurationManager 2026-03-31 22:56:29 -04:00
mat 138081bd8b fix(ha): register IServerConfigurationManager in pre-startup DI for PostgreSQL
The NpgsqlDataSource singleton factory in AddJellyfinDbContext calls
sp.GetRequiredService<IServerConfigurationManager>() to read pool settings.
During ApplyStartupMigrationAsync, only a subset of services are registered
in the startup service collection — IServerConfigurationManager was missing,
causing a fatal DI resolution failure when DatabaseType=Jellyfin-PostgreSQL.

Fix: register startupConfigurationManager as IServerConfigurationManager in
the migrationStartupServiceProvider service collection.

Also remove JELLYFIN_CONFIG_DIR=/config from Dockerfile.runtime ENV block.
When configDir == dataDir, MakeSanityCheckOrThrow writes .jellyfin-config at
the datadir root, then immediately throws because it expected .jellyfin-data.
Removing the env var lets configDir default to $JELLYFIN_DATA_DIR/config.
2026-03-31 22:56:29 -04:00
mat ea732b62d8 fix(ci): pre-build .NET on host runner, remove SDK from Docker build
DinD overlay-on-overlay throttles dotnet child container to ~11s CPU/3h wall time.
Solution: dotnet restore+publish run on native runner FS (~10min), then
Dockerfile.runtime just COPYs the pre-built publish-output/ directory.
This brings build time from 3h+ (stuck) to ~10-15 minutes total.
2026-03-31 22:56:29 -04:00
mat e7c1451b6c fix(ci): exclude integration tests — need SkiaSharp native libs and running server 2026-03-31 22:56:26 -04:00
mat a49a766424 fix(ci): export PATH in test step — each step has fresh shell on ARC runners 2026-03-31 22:56:26 -04:00
mat ddf3b9f609 fix(ci): use GITHUB_ENV not GITHUB_PATH for dotnet PATH on ARC runners
GITHUB_PATH between-step propagation is broken on summerwind ARC runner pods.
Switch to GITHUB_ENV PATH= which is reliably sourced on every step.
2026-03-31 22:56:26 -04:00
mat 9c6fab8f12 fix(ci): install .NET 10 via script to writable DOTNET_INSTALL_DIR
setup-dotnet fails on ARC runners (no permission to /usr/share/dotnet) and dotnet
is not pre-installed. Install via dotnet-install.sh into $HOME/.dotnet instead,
then add to PATH via GITHUB_PATH.
2026-03-31 22:56:26 -04:00
mat 992e591938 fix(ci): remove setup-dotnet from test workflow — ARC runners have .NET 10 pre-installed
setup-dotnet fails on self-hosted ARC runners because it cannot write to
/usr/share/dotnet (permission denied). The runner images already have .NET 10
SDK installed in DOTNET_ROOT. Remove the step entirely.
2026-03-31 22:56:26 -04:00
mat 365036f9f2 fix(ci): skip Docker-dependent PostgreSQL tests in upstream test workflow
- Add [Trait("Category", "RequiresDocker")] to all 3 PostgreSQL test classes
  (PostgreSqlMigrationTests, PostgreSqlProviderTests, PostgreSqlConcurrencyTests)
- Add --filter "Category!=RequiresDocker" to ci-tests.yml dotnet test command
  so runners without Docker don't fail on Testcontainers initialization
- Disable CodeQL workflow in fork (requires upstream org permissions + .NET 10
  CodeQL support that isn't available on our self-hosted runners)

PostgreSQL tests still run in ha-build.yml against the cluster where Docker
is available via the self-hosted ARC runners.
2026-03-31 22:56:13 -04:00
mat 93af601a5f ci: run tests and CodeQL on k3s self-hosted amd64 runners
- ci-tests.yml: drop ubuntu/macos/windows matrix → single k3s amd64 runner
- ci-codeql-analysis.yml: ubuntu-latest → k3s amd64 runner
Fork targets amd64 cluster only; self-hosted runners have Docker for Testcontainers
2026-03-31 22:56:13 -04:00
mat 894d837578 fix(ci): use version tags for docker actions 2026-03-31 22:56:13 -04:00
mat e79caccab1 fix(ci): add docker/setup-buildx-action before build step 2026-03-31 22:56:13 -04:00
mat 25b137e0ab fix(ci): switch ha-build to static AWS key auth 2026-03-31 22:56:13 -04:00
mat 75ef410437 fix(ci): trigger ha-build workflow on master branch 2026-03-31 22:56:13 -04:00
mat 849f0a06c8 fix(dockerfile): disable TreatWarningsAsErrors for Docker publish step
StyleCop analyzer violations in upstream src/Jellyfin.Extensions/ block
the Docker build when TreatWarningsAsErrors=true (from Directory.Build.props).
Disable for container image builds — StyleCop enforcement is the CI pipeline's
responsibility, not the Dockerfile's.
2026-03-31 22:56:10 -04:00
mat 2e5fb57052 fix(dockerfile): copy BannedSymbols.txt and stylecop.json into build stage
dotnet publish failed with CS2001 because these root-level analyzer config files
were not included in the Docker build context. They are referenced by
Directory.Build.props and required by StyleCop and BannedApiAnalyzers at build
time.
2026-03-31 22:56:10 -04:00
Copilot e72bde9c02 Add Jellyfin.DbMigrator SQLite-to-PostgreSQL migration tool (#14)
* Initial plan

* Add Jellyfin.DbMigrator SQLite-to-PostgreSQL migration tool

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:56:07 -04:00
Copilot 35a1ec8a5d [WIP] Add PostgreSQL integration test project for validation (#12)
* Initial plan

* Add PostgreSQL integration test project with migration, CRUD, and concurrency tests

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:55:57 -04:00
Copilot a0c38131c8 Wire NpgsqlDataSource pool into DI for PostgreSQL provider (#10)
* Initial plan

* Add NpgsqlDataSource pool wiring to DI (Issue 1.4)"

- PostgreSqlDatabaseProvider: accept NpgsqlDataSource via constructor injection, use it in Initialise()
- PostgreSqlDesignTimeJellyfinDbFactory: build NpgsqlDataSource from connection string for design-time use
- ServiceCollectionExtensions: register NpgsqlDataSource as singleton with pool params (MinPoolSize=2, MaxPoolSize=20, CommandTimeout=30) from CustomProviderOptions.Options

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:55:45 -04:00
Copilot 3bc45ff92d Add PostgreSQL EF Core design-time factory and InitialPostgreSql migration (#8)
* Initial plan

* Add PostgreSQL design-time factory and initial migration for all 29 DbSets

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

* Remove accidentally committed build artifacts from PostgreSQL provider

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:55:45 -04:00
Copilot 2d4d6d8c38 Implement PostgreSqlDatabaseProvider methods and DI registration (#6)
* Initial plan

* feat: implement PostgreSqlDatabaseProvider all methods + DI registration

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:55:45 -04:00
Copilot 9a99572354 Scaffold Jellyfin.Database.Providers.PostgreSQL project (#4)
* Initial plan

* Issue 1.1: Scaffold PostgreSQL provider project

Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ZoltyMat <177592743+ZoltyMat@users.noreply.github.com>
2026-03-31 22:55:45 -04:00
Joshua M. Boniface b2aa80ce5c Fix invalid regex comparison 2026-03-31 19:59:33 -04:00
Joshua M. Boniface ff365dae34 Fix invalid merge conflict fix 2026-03-31 19:46:47 -04:00
Jellyfin Release Bot 52aebfb7d3 Bump version to 10.11.7 2026-03-31 19:33:11 -04:00
Joshua M. Boniface 66ea1b50e6 Merge commit from fork
Fix GHSA-8fw7-f233-ffr8 with improved sanitization
2026-03-31 19:17:17 -04:00
Joshua M. Boniface 3f656ade7a Merge remote-tracking branch 'upstream/release-10.11.z' into advisory-fix-1 2026-03-31 19:16:19 -04:00
Joshua M. Boniface 8bf0d372c6 Merge commit from fork
Fix GHSA-jh22-fw8w-2v9x
2026-03-31 17:46:01 -04:00
Joshua M. Boniface 202d7b5829 Merge branch 'release-10.11.z' into advisory-fix-1 2026-03-31 17:44:59 -04:00
Joshua M. Boniface 352e4f3aba Merge commit from fork
Fix GHSA v2jv-54xj-h76w
2026-03-31 17:43:02 -04:00
Joshua M. Boniface c5f6d00c94 Merge commit from fork
Fix GHSA-j2hf-x4q5-47j3 with improved sanitization
2026-03-31 17:38:46 -04:00
Shadowghost e8d1d94436 Lock down tuner API to be admin-only 2026-03-31 16:35:15 +02:00
Shadowghost 50dc37065b Fix GHSA-jh22-fw8w-2v9x 2026-03-31 09:30:45 +02:00
Niels van Velzen 7e88b18192 Merge pull request #16522 from Bond-009/CA1810
Fix CA1810 build error
2026-03-30 18:44:15 +02:00
Bond-009 89e914c7f1 Merge pull request #16519 from jellyfin/check-h264-profile-null
Fix Null was not checked before using the H264 profile
2026-03-30 18:39:45 +02:00
Bond_009 1932ac4765 Fix CA1810 build error 2026-03-30 18:33:56 +02:00
Bond-009 ec33c74ec4 Merge pull request #16440 from Molier/fix/subtitle-extraction-flush
Remove -copyts and add -flush_packets 1 to subtitle extraction
2026-03-30 18:30:58 +02:00
nyanmisaka 2184ed1b16 Fix Null was not checked before using the H264 profile
This is rare, but not impossible.

Signed-off-by: nyanmisaka <nst799610810@gmail.com>
2026-03-30 20:51:11 +08:00
Shadowghost d3907afde7 Add additional validations 2026-03-30 10:48:51 +02:00
theguymadmax e12d933531 Revet lint fix 2026-03-30 04:21:26 -04:00
theguymadmax c0ba29d917 fix lint issue 2026-03-30 04:14:23 -04:00
Shadowghost d1fd81c382 Fix GHSA v2jv-54xj-h76w 2026-03-30 09:40:01 +02:00
Joshua M. Boniface e038045494 Fix lint 2026-03-29 19:11:40 -04:00
Joshua M. Boniface e1691e649e Merge pull request #16514 from theguymadmax/release-10.11.z-fixup 2026-03-29 19:04:53 -04:00
theguymadmax 8d28497d29 Fix lint issue 2026-03-29 18:25:57 -04:00
Joshua M. Boniface fddd4e7e6b Fix GHSA-8fw7-f233-ffr8 with improved sanitization
Co-Authored-By: Shadowghost <Ghost_of_Stone@web.de>
2026-03-29 17:30:09 -04:00
Joshua M. Boniface 0581cd6610 Fix GHSA-j2hf-x4q5-47j3 with improved sanitization
Co-Authored-By: Shadowghost <Ghost_of_Stone@web.de>
2026-03-29 17:22:14 -04:00
Joshua M. Boniface 0f1732e5f5 Merge pull request #16425 from theguymadmax/fix-metadata-backup 2026-03-29 15:22:14 -04:00
Bond-009 41c2d51d8c Merge pull request #16369 from Bond-009/skiafonts
Fix nullref ex in font handling
2026-03-29 20:27:17 +02:00
Bond-009 29b2361857 Merge pull request #16423 from nyanmisaka/fix-ffmpeg8-readrate-option
Fix readrate options in FFmpeg 8.1
2026-03-23 21:25:35 +01:00
Bond-009 ce867f9834 Merge pull request #16449 from theguymadmax/fix-collection-number
Fix NFO saver using wrong provider ID for collectionnumber
2026-03-23 19:23:34 +01:00
theguymadmax 4034bf9d7e Save collection id instead of moive id 2026-03-22 12:49:33 -04:00
Oscar 3d2658fa43 Remove -copyts and add -flush_packets 1 to subtitle extraction
-copyts is unnecessary for -c:s copy to SRT and slows extraction ~5x.
Without -flush_packets 1, ffmpeg buffers all output until exit, leaving
.srt files at 0 bytes for minutes while the player shows no subtitles.

Fixes #16438
2026-03-19 22:54:05 +01:00
theguymadmax 61b19688ff Backup default metadata location 2026-03-17 23:13:20 -04:00
theguymadmax e8d72bf6a3 Fix restore backup metadata location 2026-03-16 10:32:09 -04:00
nyanmisaka 348b14f7b7 Fix readrate options in FFmpeg 8.1
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
2026-03-16 17:58:53 +08:00
IceStormNG fda49a5a49 Apply analyzeduration and probesize for subtitle streams to improve codec parameter detection (#16293)
Apply analyzeduration and probesize for subtitle streams to improve codec parameter detection
2026-03-13 20:26:25 +01:00
Bond-009 55c00d76bb Merge pull request #16392 from nyanmisaka/fix-ffmpeg8-filter-detection
Fix filter detection in FFmpeg 8.1
2026-03-13 20:24:18 +01:00
nyanmisaka 519d2113eb Fix filter detection in FFmpeg 8.1
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
2026-03-11 17:46:07 +08:00
Bond_009 f34f6b6941 Fix nullref ex in font handling
Don't add fallback fonts if they are null
The default font can still be null, however unlikely
2026-03-08 12:46:25 +01:00
Joshua M. Boniface 6864e108b8 Merge pull request #16257 from lowbit/fix/subtitle-empty-file-cache 2026-03-07 00:53:56 -05:00
Bond-009 09ba04662a Merge pull request #16341 from crimsonspecter/fix-fractional-hls-time-for-remux
Fix hls segment length adjustment for remuxed content
2026-03-06 22:39:59 +01:00
crimsonspecter 9cd2418095 Fix: don't apply segment length adjustment for remuxed content 2026-03-04 20:27:06 +01:00
Bond-009 b6a96513de Merge pull request #16253 from theguymadmax/use-backupdatabase
Checkpoint WAL before moving library.db in migration
2026-02-28 10:08:36 +01:00
Bond-009 ca57166e95 Merge pull request #16204 from MBR-0001/fix-bad-subtitle-settings
Fix broken library subtitle download settings
2026-02-28 10:08:20 +01:00
MBR#0001 33496c1693 Fix broken library subtitle download settings 2026-02-26 19:54:24 +01:00
Bond-009 b65daeca0b Merge pull request #16150 from dfederm/nullref-season-series
Fix nullref in Season.GetEpisodes when the season is detached from a series
2026-02-22 13:49:41 +01:00
David Federman 286cc6d720 Fix nullref in Season.GetEpisodes when the season is detached from a series 2026-02-20 22:56:30 -08:00
Andrew Rabert aa4f09c799 Mitigate pull_request_target privilege escalation
Hotfix — replaces pull_request_target with pull_request to stop
granting write permissions and secrets to fork PRs. Some workflows
will break; can be fixed properly later.
2026-02-20 19:10:40 -05:00
rijads afd3c0d9f3 Fix subtitle extraxtion caching empty files 2026-02-19 17:53:47 +01:00
theguymadmax 5597d8e1a7 Checkpoint wal 2026-02-18 15:11:57 -05:00
theguymadmax 0166362258 Use BackupDatabase() instead of File.Move in library.db migration 2026-02-17 22:56:45 -05:00
Bond-009 58c330b63d Merge pull request #16226 from dfederm/fix/16134-migration-unique-constraint
Deduplicate provider IDs during MigrateLibraryDb migration
2026-02-14 11:46:43 +01:00
Bond-009 be71295693 Merge pull request #16227 from dfederm/fix/16149-watch-state-episode-replace
Reattach user data after item removal during library scan
2026-02-14 11:45:51 +01:00
Bond-009 8cd3090cee Merge pull request #16231 from theguymadmax/skip-image-for-empty-folders
Skip image checks for empty folders
2026-02-14 11:43:24 +01:00
David Federman 7bf08daeec Reattach user data after removing items during library scan
When items are removed during a library scan, their user data is
detached to a placeholder. If a replacement item already exists
(e.g., a new version of the same episode was added before the old
file was deleted), the user data would be stranded in the placeholder
because the replacement item's initial ReattachUserDataAsync call
happened before the old item was deleted.

This fix checks for remaining valid children that share user data
keys with removed items and reattaches any detached user data to them.

Fixes #16149
2026-02-12 20:38:28 -08:00
David Federman 290463fe7b Fix migration UNIQUE constraint on BaseItemProviders
Deduplicate ProviderIds by ProviderId during MigrateLibraryDb migration
to prevent UNIQUE constraint violations when legacy data contains
duplicate provider entries for the same item.

Fixes #16134
2026-02-12 20:37:26 -08:00
theguymadmax 1b2d9c100a Skip image checks for empty folders 2026-02-12 18:04:27 -05:00
Bond-009 caa05c1bf2 Merge pull request #16116 from saltpi/bugfix
Fix TMDB image URLs missing size parameter
2026-02-02 20:48:21 +01:00
Niels van Velzen a37ead86df Merge pull request #16098 from theguymadmax/fix-random-sort
Fix random sort returning duplicate items
2026-01-27 11:31:41 +01:00
Niels van Velzen e65aff8bc6 Merge pull request #16109 from nielsvanvelzen/ws-session-info-dto-backport
Fix SessionInfoWebSocketListener not using SessionInfoDto
2026-01-27 11:31:15 +01:00
endpne 9734494eb6 Fix TMDB image URLs missing size parameter 2026-01-26 13:25:03 +08:00
Niels van Velzen d41e302418 Fix SessionInfoWebSocketListener not using SessionInfoDto 2026-01-25 21:21:48 +01:00
theguymadmax 80ba517294 Fix random sort returning duplicate items 2026-01-24 13:48:05 -05:00
MarcoCoreDuo 95d08b264f Rehydrate cached UserData after reattachment (#16071) 2026-01-22 17:43:05 -07:00
IceStormNG 893a849f28 Slightly adjust segment length for fractional framerates (#16053)
Co-authored-by: Carsten Braun <carsten.braun@braun-cloud.de>
2026-01-22 17:41:51 -07:00
theguymadmax 673f617994 Fix TMDB crew department mapping (#16066) 2026-01-22 17:40:35 -07:00
theguymadmax 644327eb76 Revert hidden directory ignore pattern (#16077) 2026-01-22 17:39:55 -07:00
Jellyfin Release Bot 10662e75e4 Bump version to 10.11.6 2026-01-18 20:02:59 -05:00
Joshua M. Boniface a2b1936e73 Merge pull request #15816 from theguymadmax/preserve-artist-order
Fix artist display order
2026-01-18 19:48:17 -05:00
theguymadmax 2df546af6d Deduplicate using Distinct 2026-01-18 18:16:45 -05:00
Claus Vium 338b480217 Merge pull request #16046 from theguymadmax/restore-weekly-images
Restore weekly refresh for library folder images
2026-01-18 16:36:45 +01:00
theguymadmax 2943bb6fdd Restore collection folder image refresh 2026-01-18 01:51:51 -05:00
theguymadmax 94edcbd2d1 Fix artist ordering DtoServices 2026-01-17 10:14:41 -05:00
theguymadmax a8d1cdefac Address review comments 2026-01-17 10:14:41 -05:00
Tim Eisele a518160a6f Prioritize better matches on search (#15983) 2026-01-16 19:05:46 -07:00
Tim Eisele b56de6493f Be more strict about PersonType assignments (#15872) 2026-01-16 19:03:13 -07:00
theguymadmax 093cfc3f3b Trim music artist names (#15808) 2026-01-16 18:51:48 -07:00
theguymadmax 49775b1f6a Fix birthplace not saving correctly (#16020) 2026-01-16 18:47:40 -07:00
Collin T Swisher 22d593b8e9 Add mblink creation logic to library update endpoint. (#15965) 2026-01-16 18:47:04 -07:00
theguymadmax 2cb7fb52d2 Skip hidden directories and .ignore paths in library monitoring (#16029) 2026-01-16 18:45:19 -07:00
Joshua M. Boniface 8433b6d8a4 Merge pull request #15899 from MarcoCoreDuo/fix-watch-state-not-kept
Fix watched state not kept on Media replace/rename
2026-01-16 16:40:36 -05:00
Bond-009 32d2414de0 Merge pull request #15950 from theguymadmax/revert-sort-index-number
Revert "always sort season by index number"
2026-01-09 18:38:23 +01:00
Bond-009 317a3a47c3 Merge pull request #15961 from theguymadmax/fix-bad-plugin-url
Fix crash when plugin repository has an invalid URL
2026-01-09 18:24:48 +01:00
theguymadmax 845b8cdc8f Fix crash when plugin repository has an invalid URL 2026-01-06 11:57:25 -05:00
theguymadmax c86f6439c5 Revert "always sort season by index number"
This reverts commit e16ea7b236.
2026-01-05 11:06:25 -05:00
theguymadmax 559e0088e5 Fix tag inheritance for Continue Watching queries (#15931) 2026-01-04 11:20:34 -07:00
MarcoCoreDuo adaca95590 make db context creation async 2025-12-31 07:43:07 +01:00
MarcoCoreDuo 09a1c31fa3 Refactor ReattachUserData methods to be asynchronous 2025-12-31 03:06:07 +01:00
MarcoCoreDuo e4b82025b8 move reattaching user data to own function and call it only after fetching metadata for the first time 2025-12-30 22:04:59 +01:00
Collin T Swisher 78e3702cb0 Fix playlist item de-duplication (#15858) 2025-12-24 07:50:15 -07:00
Bond-009 01b20d3b75 Merge pull request #15833 from nyanmisaka/fix-h264-av1-sdr-hls-fallback
Fix missing H.264 and AV1 SDR fallbacks in HLS playlist
2025-12-24 10:28:33 +01:00
Tim Eisele 156761405e Prefer US rating on fallback (#15793) 2025-12-19 20:41:09 -07:00
Claus Vium 1805f2259f add CultureDto cache (#15826) 2025-12-19 20:38:54 -07:00
Nyanmisaka 4c587776d6 Fix the use of HWA in unsupported H.264 Hi422P/Hi444PP (#15819) 2025-12-19 19:58:56 -07:00
gnattu 8379b4634a Enforce more strict webm check (#15807) 2025-12-19 19:57:08 -07:00
Nyanmisaka 9470439cfa Fix video lacking SAR and DAR are marked as anamorphic (#15834) 2025-12-19 19:54:48 -07:00
gnattu 18096e48e0 Use hvc1 codectag for Dolby Vision 8.4 (#15835) 2025-12-19 19:53:28 -07:00
nyanmisaka f2d0ac7b28 Fix missing H.264 and AV1 SDR fallbacks in HLS playlist
Previously, if HEVC encoding was disabled on the server,
SDR fallbacks would not be provided.

Signed-off-by: nyanmisaka <nst799610810@gmail.com>
2025-12-19 20:33:24 +08:00
theguymadmax 2ccf08f547 Fix artist display order 2025-12-17 01:09:13 -05:00
Jellyfin Release Bot 1e27f460fe Bump version to 10.11.5 2025-12-14 21:44:14 -05:00
Andrew Rabert 4cdd8c8233 Fix unnecessary database JOINs in ApplyNavigations (#15666) 2025-12-13 10:58:08 -07:00
Tim Eisele 6e60634c9f Skip invalid ignore rules (#15746) 2025-12-13 08:39:49 -07:00
theguymadmax 12c5d6b636 Fix backdrop images being deleted when stored with media (#15766) 2025-12-13 08:29:17 -07:00
theguymadmax b617c62f8e Fix NullReferenceException in ApplyOrder method (#15768) 2025-12-13 08:28:31 -07:00
Nyanmisaka 035b5895b0 Fix AV1 decoding hang regression on RK3588 (#15776) 2025-12-13 08:27:29 -07:00
theguymadmax 22da5187c8 Fix collection display order (#15767) 2025-12-13 08:27:01 -07:00
theguymadmax 5804d6840c Fix parental rating comparison with sub-scores (#15786) 2025-12-13 08:25:48 -07:00
Bond-009 b50ce1ad6b Merge pull request #15752 from Collin-Swish/fix-name-case-insensitivity
Fix case sensitivity edge case
2025-12-12 21:39:22 +01:00
Bond-009 481ee03f35 Merge pull request #15757 from theguymadmax/fix-trickplays-for-alt-versions
Fix trickplay images using wrong item on alternate versions
2025-12-12 21:31:52 +01:00
Bond-009 d91adb5d54 Merge pull request #15662 from SapientGuardian/issue15661
Fix blocking in async context in LimitedConcurrencyLibraryScheduler
2025-12-10 20:37:57 +01:00
theguymadmax ef7f138a4e Fix trickplay images using wrong item on alternate versions 2025-12-09 14:21:09 -05:00
Collin Swisher 2e8d9a311b Fix case sensitivity edge case 2025-12-08 17:41:48 -06:00
gnattu 4c5a3fbff3 Use original name for MusicAritist matching (#15689) 2025-12-05 19:30:02 -07:00
liszto 636908fc4d Fix thumbnails not being deleted from temp folder 2025-12-05 19:29:54 -07:00
Tim Eisele 997362fc97 Backport dependency updates (#15723) 2025-12-05 19:27:30 -07:00
Noah Potash c5147341e3 Fixes 15661. Replace BlockingCollection with Channel in LimitedConcurrencyLibraryScheduler to prevent blocking in an asynchronous context. 2025-12-03 21:50:08 -05:00
Noah Potash ca33bcebf0 Add SapientGuardian to CONTRIBUTORS.md 2025-12-03 21:27:26 -05:00
Ivan Kara d32f487e8e Fix symlinked file size (#15681) 2025-12-03 19:04:59 -07:00
theguymadmax fb65f8f853 Fix ItemAdded event triggering when updating metadata (#15680) 2025-12-03 19:02:55 -07:00
martenumberto 2a0b90e385 Fix: Add .ts fallback for video streams to prevent crash (#15690) 2025-12-03 19:02:39 -07:00
myzhysz dde70fd8a2 Fix stack overflow while scanning (#15698) 2025-12-03 19:02:04 -07:00
Niels van Velzen 98d1d0cb35 Merge pull request #15670 from nyanmisaka/fix-mjpeg-rk3576
Fix the empty output of trickplay on RK3576
2025-12-02 13:48:51 +01:00
Jellyfin Release Bot ba76a8f3ad Bump version to 10.11.4 2025-11-30 21:33:32 -05:00
Anthony Lavado 8cd5652157 Merge pull request #15672 from jellyfin/openapi-cache-z
Cache OpenApi document generation
2025-11-30 21:22:29 -05:00
crobibero 8aff4227d9 Implement caching for OpenAPI document 2025-11-30 09:19:19 -07:00
nyanmisaka 026f7472cb Fix the empty output of trickplay on RK3576
Signed-off-by: nyanmisaka <nst799610810@gmail.com>
2025-11-30 21:38:47 +08:00
MBR-0001 daca285568 Revert "Localization/iso6392.txt: change pob and pop" (#15555) 2025-11-23 19:20:29 +01:00
theguymadmax fbb9a0b2c7 Fix ResolveLinkTarget crashing on exFAT drives (#15568) 2025-11-21 21:14:39 -07:00
Ziyuan Qu 29b3aa8543 Add hidden file check in bdInfo (#15582) 2025-11-21 21:14:30 -07:00
theguymadmax 94f3725208 Fix isMovie filter logic (#15594) 2025-11-21 21:14:03 -07:00
theguymadmax 0ee81e87be Fix locked fields on not saving (#15564) 2025-11-19 17:02:53 +01:00
theguymadmax c491a918c2 Save item to database before providers run to prevent FK constraint errors (#15563) 2025-11-19 17:01:13 +01:00
gnattu 1e7e46cb82 Prevent copying HDR streams when only SDR is supported (#15556) 2025-11-18 18:37:35 -07:00
theguymadmax 5ae444d96d Fix NullReferenceException in filesystem path comparison (#15548) 2025-11-18 18:37:09 -07:00
gnattu ee7ad83427 Restrict first video frame probing to file protocol (#15557) 2025-11-18 18:36:59 -07:00
Jellyfin Release Bot 921d7d3364 Bump version to 10.11.3 2025-11-16 17:40:07 -05:00
theguymadmax f8e012582a Fix movie titles using folder name when NFOs saver is enabled (#15529) 2025-11-16 13:59:58 -07:00
theguymadmax def5956cd1 Fix tmdbid not detected in single movie folder (#14955) 2025-11-16 13:36:35 -07:00
theguymadmax abfbaca336 Fix series DateLastMediaAdded not updating when new episodes are added (#15472) 2025-11-16 13:35:43 -07:00
theguymadmax 6566188e45 Add 1 minute tolerance for NFO change detection (#15514) 2025-11-15 08:39:25 -07:00
theguymadmax 078f9584ed Fix playlist DateCreated and DateLastMediaAdded not being set (#15508) 2025-11-14 15:19:40 -07:00
Iksas ee34c75386 fix missing font extraction for certain transcoding settings (#15502) 2025-11-13 18:30:18 -07:00
theguymadmax e8150428b6 Fix .ignore handling for directories (#15501) 2025-11-13 18:23:18 -07:00
theguymadmax 4b38e35bbb Remove InheritedTags and update tag filtering logic (#15493) 2025-11-13 18:23:03 -07:00
Huo Jiacheng 435bb14bb2 Fix gitignore-style not working properly on windows. (#15487) 2025-11-12 19:43:13 -07:00
theguymadmax 2e5ced5098 Improve season folder parsing (#15404) 2025-11-12 17:36:57 -07:00
Bond-009 f4a846aa4d Don't error out when searching for marker files fails (#15466)
Fixes #15445
2025-11-11 15:45:47 -07:00
Joshua M. Boniface 7c1063177f Merge pull request #15462 from theguymadmax/fix-exception-for-empty-strm-files
Fix NullReferenceException in GetPathProtocol when path is null
2025-11-10 19:30:38 -05:00
Joshua M. Boniface 5878b1ffc5 Merge pull request #15468 from Bond-009/carefulWithLastMinChanges
Check if target exists before trying to follow it
2025-11-10 19:12:24 -05:00
Bond_009 3c3c2aee0d Check if target exists before trying to follow it
Exception got caught in ManagedFileSystem and wrong file info got returned
2025-11-10 23:19:17 +01:00
theguymadmax 511223aac4 Fix NullReferenceException in GetPathProtocol when path is null 2025-11-10 02:30:49 -05:00
Mikal S. 3b2d64995a Resolve symlinks for static media source infos (#15263) 2025-11-09 09:45:02 -07:00
theguymadmax 13c4517a66 Fix collection grouping in mixed libraries (#15373) 2025-11-09 09:35:50 -07:00
theguymadmax 177b6464ca Don't clear baseitemids (#15446) 2025-11-09 09:22:09 -07:00
Bond-009 5a9a8363f4 Merge pull request #15441 from IceStormNG/fix-nullreference-role-null-10.11
Fix System.NullReferenceException when people's role is null (10.11.z)
2025-11-08 18:25:03 +01:00
theguymadmax 49efd68fc7 Invalidate parent folder's cache on deletion/creation (#15423) 2025-11-08 08:30:04 -07:00
Carsten Braun 90a8a26c6e Copy-Pasting is sometimes hard.... 2025-11-08 15:00:11 +01:00
Carsten Braun 002c83e6f5 Fix NullReferenceExceltop when role is null. 2025-11-08 14:32:14 +01:00
theguymadmax 7222910b05 Fix filters to use SortName (#15381) 2025-11-07 18:21:41 -07:00
Bond-009 097cb87f6f Don't enforce a minimum amount of free space for the tmp and log dirs (#15390) 2025-11-07 18:21:10 -07:00
JPVenson 91c3b1617e Fixed missing sort argument (#15413) 2025-11-07 18:20:42 -07:00
theguymadmax 8f71922734 Fix item count display for collapsed items (#15380) 2025-11-07 18:20:10 -07:00
Niels van Velzen d140630208 Update branding in Swagger page (#15422) 2025-11-07 18:19:30 -07:00
theguymadmax 63a3e55297 Fix search terms using diacritics (#15435) 2025-11-07 18:18:24 -07:00
evanreichard c2e5081d64 feat(sqlite): add timeout config (#15369) 2025-11-07 18:17:43 -07:00
Jellyfin Release Bot 4187c6f620 Bump version to 10.11.2 2025-11-02 21:28:56 -05:00
Tim Eisele e7dbb3afec Skip too large extracted season numbers (#15326) 2025-11-02 09:11:48 -07:00
vinnyspb f994dd6211 Update file size when refreshing metadata (#15325) 2025-11-01 14:18:19 -06:00
Cody Robibero da254ee968 return instead of break, add check to more migrations (#15322) 2025-11-01 14:17:22 -06:00
Bill Thornton 4ad3141875 Update password reset to always return the same response structure (#15254) 2025-11-01 14:17:09 -06:00
evanreichard b5f0199a25 fix: in optimistic locking, key off table is locked (#15328) 2025-11-01 14:15:26 -06:00
Nyanmisaka 6bf88c049e Ignore initial delay in audio-only containers (#15247) 2025-10-29 20:40:28 -06:00
Jellyfin Release Bot 40a33da2a5 Bump version to 10.11.1 2025-10-26 22:02:09 -04:00
Joshua M. Boniface 3596fc0693 Fix bump_version to handle spaced filename 2025-10-26 21:50:38 -04:00
Jellyfin Release Bot 93824dad97 Bump version to 10.11.1 2025-10-26 21:41:27 -04:00
Tim Eisele e5656af1f2 Improve symlink handling (#15209) 2025-10-26 15:10:13 -06:00
Niels van Velzen c127c10458 Merge pull request #15225 from Bond-009/z440ATL
Update dependency z440.atl.core to 7.6.0
2025-10-26 18:50:04 +01:00
Tim Eisele 7d1824ea27 Fix pagination and sorting for folders (#15187) 2025-10-26 11:34:11 -06:00
Cody Robibero 2966d27c97 Skip invalid database migration (#15212) 2025-10-26 11:34:04 -06:00
Ivan Kara 618ec4543e Add season number fallback for OMDB and TMDB plugins (#15113) 2025-10-26 11:33:55 -06:00
Cody Robibero 0e4031ae52 Skip extracting directory entry when restoring (#15196) 2025-10-26 11:33:47 -06:00
CeruleanRed 442af96ed9 Only save chapters that are within the runtime of the video file (#15176) 2025-10-26 10:37:16 -06:00
JJBlue a305204cfa Skip extracted files in migration if bad timestamp or no access (#15220)
Fixes #15024
2025-10-26 10:30:43 -06:00
theguymadmax 75f472e6a7 Normalize paths in database queries (#15217) 2025-10-26 10:30:12 -06:00
Bond_009 cc32e8f7cb Update dependency z440.atl.core to 7.6.0 2025-10-26 15:16:08 +01:00
MBR-0001 14b3085ff1 Fix Has(Imdb/Tmdb/Tvdb)Id checks (#15126) 2025-10-25 16:00:55 -06:00
Cody Robibero 5691eee4f1 Prefer filting by package id instead of name (#15197) 2025-10-25 09:37:09 -06:00
theguymadmax 1520a697ad Play selected song first with instant mix (#15133) 2025-10-25 09:33:11 -06:00
Cody Robibero 81b8b0ca4a Add the transcode marker during startup instead of first transcode (#15194) 2025-10-25 09:32:15 -06:00
Cody Robibero ac3fa3c376 Clean up backup service (#15170) 2025-10-24 17:57:34 -06:00
Tim Eisele 7a1c1cd342 Skip extracted files in migration if bad timestamp or no access (#15112) 2025-10-24 17:57:19 -06:00
gnattu 70c32a26fa Make priority class setting more robust (#15177) 2025-10-24 17:57:02 -06:00
Cody Robibero 2b94bb54aa Fix xml formatter (#15164) 2025-10-24 17:56:38 -06:00
Bond-009 0a6e8146be Lower required tmp dir size to 512MiB (#15098) 2025-10-23 16:38:27 -06:00
theguymadmax 305b0fdca3 Make season paths case-insensitive (#15102) 2025-10-23 16:38:06 -06:00
theguymadmax d738386fe2 Fix LiveTV images not saving to database (#15083) 2025-10-23 16:37:55 -06:00
Tim Eisele ca830d5be7 Speed-up trickplay migration (#15054) 2025-10-23 16:37:47 -06:00
theguymadmax a5bc4524d8 Optimize artist query (#15087) 2025-10-23 16:37:29 -06:00
Nyanmisaka 175ee12bbc Fix videos with cropping metadata are probed as anamorphic (#15144) 2025-10-23 16:31:11 -06:00
Nyanmisaka a725220c21 Reject stream copy of HDR10+ video if the client does not support HDR10 (#15072) 2025-10-21 17:20:56 -06:00
gnattu a245605152 Log the message more clear when network manager is not ready (#15055) 2025-10-21 17:18:26 -06:00
Tim Eisele f4a53209f4 Skip invalid keyframe cache data (#15032) 2025-10-21 17:17:56 -06:00
Jellyfin Release Bot 877251bcae Bump version to 10.11.0 2025-10-19 20:45:12 -04:00
233 changed files with 25225 additions and 1932 deletions
-44
View File
@@ -1,44 +0,0 @@
name: "CodeQL"
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
schedule:
- cron: '24 2 * * 4'
permissions:
contents: read
security-events: write
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
language: [ 'csharp' ]
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: '10.0.x'
- name: Initialize CodeQL
uses: github/codeql-action/init@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
with:
languages: ${{ matrix.language }}
queries: +security-extended
- name: Autobuild
uses: github/codeql-action/autobuild@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
-159
View File
@@ -1,159 +0,0 @@
name: ABI Compatibility
on:
pull_request:
permissions: {}
jobs:
abi-head:
name: ABI - HEAD
runs-on: ubuntu-latest
permissions: read-all
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
- name: Setup .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: '10.0.x'
- name: Build
run: |
dotnet build Jellyfin.Server -o ./out
- name: Upload Head
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: abi-head
retention-days: 14
if-no-files-found: error
path: out/
abi-base:
name: ABI - BASE
if: ${{ github.base_ref != '' }}
runs-on: ubuntu-latest
permissions: read-all
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Setup .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: '10.0.x'
- name: Checkout common ancestor
env:
HEAD_REF: ${{ github.head_ref }}
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}
git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules upstream +refs/heads/*:refs/remotes/upstream/* +refs/tags/*:refs/tags/*
ANCESTOR_REF=$(git merge-base upstream/${{ github.base_ref }} origin/$HEAD_REF)
git checkout --progress --force $ANCESTOR_REF
- name: Build
run: |
dotnet build Jellyfin.Server -o ./out
- name: Upload Head
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: abi-base
retention-days: 14
if-no-files-found: error
path: out/
abi-diff:
permissions:
pull-requests: write # to create or update comment (peter-evans/create-or-update-comment)
name: ABI - Difference
if: ${{ github.event_name == 'pull_request' }}
runs-on: ubuntu-latest
needs:
- abi-head
- abi-base
steps:
- name: Download abi-head
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: abi-head
path: abi-head
- name: Download abi-base
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: abi-base
path: abi-base
- name: Setup ApiCompat
run: |
dotnet tool install --global Microsoft.DotNet.ApiCompat.Tool
- name: Run ApiCompat
id: diff
run: |
{
echo 'body<<EOF'
for file in Jellyfin.Data.dll MediaBrowser.Common.dll MediaBrowser.Controller.dll MediaBrowser.Model.dll Emby.Naming.dll Jellyfin.Extensions.dll Jellyfin.MediaEncoding.Keyframes.dll Jellyfin.Database.Implementations.dll; do
COMPAT_OUTPUT="$( { apicompat --left ./abi-base/${file} --right ./abi-head/${file}; } 2>&1 || true )"
if [ "APICompat ran successfully without finding any breaking changes." != "${COMPAT_OUTPUT}" ]; then
printf "\n${file}\n${COMPAT_OUTPUT}\n"
fi
done
echo EOF
} >> $GITHUB_OUTPUT
- name: Find difference comment
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
id: find-comment
with:
issue-number: ${{ github.event.pull_request.number }}
direction: last
body-includes: abi-diff-workflow-comment
- name: Reply or edit difference comment (changed)
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
if: ${{ steps.diff.outputs.body != '' }}
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.find-comment.outputs.comment-id }}
edit-mode: replace
token: ${{ secrets.JF_BOT_TOKEN }}
body: |
<!--abi-diff-workflow-comment-->
<details>
<summary>ABI Difference</summary>
```
${{ steps.diff.outputs.body }}
```
</details>
- name: Reply or edit difference comment (unchanged)
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
if: ${{ steps.diff.outputs.body == '' && steps.find-comment.outputs.comment-id != '' }}
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.find-comment.outputs.comment-id }}
edit-mode: replace
token: ${{ secrets.JF_BOT_TOKEN }}
body: |
<!--abi-diff-workflow-comment-->
<details>
<summary>ABI Difference</summary>
No changes to the ABI found. See history of this comment for previous changes.
</details>
-25
View File
@@ -1,25 +0,0 @@
name: Format
on:
push:
branches:
- master
# Run formatter against the forked branch, but
# do not allow access to secrets
# https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflows-in-forked-repositories
pull_request:
env:
SDK_VERSION: "10.0.x"
jobs:
format-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: ${{ env.SDK_VERSION }}
- name: Run DotNet Format
run: dotnet format --verify-no-changes --verbosity minimal
-45
View File
@@ -1,45 +0,0 @@
name: Tests
on:
push:
branches:
- master
# Run tests against the forked branch, but
# do not allow access to secrets
# https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflows-in-forked-repositories
pull_request:
env:
SDK_VERSION: "10.0.x"
jobs:
run-tests:
strategy:
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
fail-fast: false
runs-on: "${{ matrix.os }}"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: ${{ env.SDK_VERSION }}
- name: Run DotNet CLI Tests
run: >
dotnet test Jellyfin.sln
--configuration Release
--collect:"XPlat Code Coverage"
--settings tests/coverletArgs.runsettings
--verbosity minimal
- name: Merge code coverage results
uses: danielpalme/ReportGenerator-GitHub-Action@d3ebf1f760f7d8ab92cc44d9bcfee7ad73722a31 # v5.5.11
with:
reports: "**/coverage.cobertura.xml"
targetdir: "merged/"
reporttypes: "Cobertura"
# TODO - which action / tool to use to publish code coverage results?
# - name: Publish code coverage results
-63
View File
@@ -1,63 +0,0 @@
name: Commands
on:
issue_comment:
types:
- created
- edited
pull_request:
types:
- labeled
- synchronize
permissions: {}
jobs:
rebase:
name: Rebase
if: github.event.issue.pull_request != '' && contains(github.event.comment.body, '@jellyfin-bot rebase') && github.event.comment.author_association == 'MEMBER'
runs-on: ubuntu-latest
steps:
- name: Notify as seen
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
token: ${{ secrets.JF_BOT_TOKEN }}
comment-id: ${{ github.event.comment.id }}
reactions: '+1'
- name: Checkout the latest code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
token: ${{ secrets.JF_BOT_TOKEN }}
fetch-depth: 0
- name: Automatic Rebase
uses: cirrus-actions/rebase@b87d48154a87a85666003575337e27b8cd65f691 # 1.8
env:
GITHUB_TOKEN: ${{ secrets.JF_BOT_TOKEN }}
rename:
name: Rename
if: contains(github.event.comment.body, '@jellyfin-bot rename')
runs-on: ubuntu-latest
steps:
- name: pull in script
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: jellyfin/jellyfin-triage-script
- name: install python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.14'
cache: 'pip'
- name: install python packages
run: pip install -r rename/requirements.txt
- name: run rename script
run: python3 rename.py
working-directory: ./rename
env:
GH_TOKEN: ${{ secrets.JF_BOT_TOKEN }}
GH_REPO: ${{ github.repository }}
ISSUE: ${{ github.event.issue.number }}
COMMENT_ID: ${{ github.event.comment.id }}
-35
View File
@@ -1,35 +0,0 @@
name: Stale Issue Labeler
on:
schedule:
- cron: '30 1 * * *'
workflow_dispatch:
permissions:
issues: write
pull-requests: write
actions: write
jobs:
issues:
name: Check for stale issues
runs-on: ubuntu-latest
if: ${{ contains(github.repository, 'jellyfin/') }}
steps:
- uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
with:
repo-token: ${{ secrets.JF_BOT_TOKEN }}
ascending: true
days-before-stale: 120
days-before-pr-stale: -1
days-before-close: 21
days-before-pr-close: -1
operations-per-run: 500
exempt-issue-labels: regression,security,roadmap,future,feature,enhancement,confirmed
stale-issue-label: stale
stale-issue-message: |-
This issue has gone 120 days without an update and will be closed within 21 days if there is no new activity. To prevent this issue from being closed, please confirm the issue has not already been fixed by providing updated examples or logs.
If you have any questions you can use one of several ways to [contact us](https://jellyfin.org/contact).
close-issue-message: |-
This issue was closed due to inactivity.
@@ -1,32 +0,0 @@
name: Check Issue Template
on:
issues:
types:
- opened
jobs:
check_issue:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: pull in script
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: jellyfin/jellyfin-triage-script
- name: install python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.14'
cache: 'pip'
- name: install python packages
run: pip install -r main-repo-triage/requirements.txt
- name: check and comment issue
working-directory: ./main-repo-triage
run: python3 single_issue_gha.py
env:
GH_TOKEN: ${{ secrets.JF_BOT_TOKEN }}
GH_REPO: ${{ github.repository }}
ISSUE: ${{ github.event.issue.number }}
-44
View File
@@ -1,44 +0,0 @@
name: OpenAPI Generate
on:
workflow_call:
inputs:
ref:
required: true
type: string
repository:
required: true
type: string
artifact:
required: true
type: string
permissions:
contents: read
jobs:
main:
name: Main
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref }}
repository: ${{ inputs.repository }}
- name: Configure .NET
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0
with:
dotnet-version: '10.0.x'
- name: Create File
run: dotnet test tests/Jellyfin.Server.Integration.Tests/Jellyfin.Server.Integration.Tests.csproj -c Release --filter Jellyfin.Server.Integration.Tests.OpenApiSpecTests
- name: Upload Artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ inputs.artifact }}
path: tests/Jellyfin.Server.Integration.Tests/bin/Release/net10.0/openapi.json
retention-days: 14
if-no-files-found: error
-140
View File
@@ -1,140 +0,0 @@
name: OpenAPI Publish
on:
push:
branches:
- master
tags:
- 'v*'
jobs:
publish-openapi:
name: OpenAPI - Publish Artifact
uses: ./.github/workflows/openapi-generate.yml
permissions:
contents: read
with:
ref: ${{ github.sha }}
repository: ${{ github.repository }}
artifact: openapi-head
publish-unstable:
name: OpenAPI - Publish Unstable Spec
if: ${{ github.event_name != 'pull_request' && !startsWith(github.ref, 'refs/tags/v') && contains(github.repository_owner, 'jellyfin') }}
runs-on: ubuntu-latest
needs:
- publish-openapi
steps:
- name: Set unstable dated version
id: version
run: |-
echo "JELLYFIN_VERSION=$(date +'%Y%m%d%H%M%S')" >> $GITHUB_ENV
- name: Download openapi-head
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: openapi-head
path: openapi-head
- name: Upload openapi.json (unstable) to repository server
uses: appleboy/scp-action@ff85246acaad7bdce478db94a363cd2bf7c90345 # v1.0.0
with:
host: "${{ secrets.REPO_HOST }}"
username: "${{ secrets.REPO_USER }}"
key: "${{ secrets.REPO_KEY }}"
source: openapi-head/openapi.json
strip_components: 1
target: "/srv/incoming/openapi/unstable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}"
- name: Move openapi.json (unstable) into place
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
with:
host: "${{ secrets.REPO_HOST }}"
username: "${{ secrets.REPO_USER }}"
key: "${{ secrets.REPO_KEY }}"
debug: false
script: |
if ! test -d /run/workflows; then
sudo mkdir -p /run/workflows
sudo chown ${{ secrets.REPO_USER }} /run/workflows
fi
(
flock -x -w 300 200 || exit 1
TGT_DIR="/srv/repository/main/openapi"
LAST_SPEC="$( ls -lt ${TGT_DIR}/unstable/ | grep 'jellyfin-openapi' | head -1 | awk '{ print $NF }' )"
# If new and previous spec don't differ (diff retcode 0), remove incoming and finish
if diff /srv/incoming/openapi/unstable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}/openapi.json ${TGT_DIR}/unstable/${LAST_SPEC} &>/dev/null; then
rm -r /srv/incoming/openapi/unstable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}
exit 0
fi
# Move new spec into place
sudo mv /srv/incoming/openapi/unstable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}/openapi.json ${TGT_DIR}/unstable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}.json
# Delete previous jellyfin-openapi-unstable_previous.json
sudo rm ${TGT_DIR}/jellyfin-openapi-unstable_previous.json
# Move current jellyfin-openapi-unstable.json symlink to jellyfin-openapi-unstable_previous.json
sudo mv ${TGT_DIR}/jellyfin-openapi-unstable.json ${TGT_DIR}/jellyfin-openapi-unstable_previous.json
# Create new jellyfin-openapi-unstable.json symlink
sudo ln -s unstable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}.json ${TGT_DIR}/jellyfin-openapi-unstable.json
# Check that the previous openapi unstable spec link is correct
if [[ "$( readlink ${TGT_DIR}/jellyfin-openapi-unstable_previous.json )" != "unstable/${LAST_SPEC}" ]]; then
sudo rm ${TGT_DIR}/jellyfin-openapi-unstable_previous.json
sudo ln -s unstable/${LAST_SPEC} ${TGT_DIR}/jellyfin-openapi-unstable_previous.json
fi
) 200>/run/workflows/openapi-unstable.lock
publish-stable:
name: OpenAPI - Publish Stable Spec
if: ${{ startsWith(github.ref, 'refs/tags/v') && contains(github.repository_owner, 'jellyfin') }}
runs-on: ubuntu-latest
needs:
- publish-openapi
steps:
- name: Set version number
id: version
run: |-
echo "JELLYFIN_VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
- name: Download openapi-head
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: openapi-head
path: openapi-head
- name: Upload openapi.json (stable) to repository server
uses: appleboy/scp-action@ff85246acaad7bdce478db94a363cd2bf7c90345 # v1.0.0
with:
host: "${{ secrets.REPO_HOST }}"
username: "${{ secrets.REPO_USER }}"
key: "${{ secrets.REPO_KEY }}"
source: openapi-head/openapi.json
strip_components: 1
target: "/srv/incoming/openapi/stable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}"
- name: Move openapi.json (stable) into place
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
with:
host: "${{ secrets.REPO_HOST }}"
username: "${{ secrets.REPO_USER }}"
key: "${{ secrets.REPO_KEY }}"
debug: false
script: |
if ! test -d /run/workflows; then
sudo mkdir -p /run/workflows
sudo chown ${{ secrets.REPO_USER }} /run/workflows
fi
(
flock -x -w 300 200 || exit 1
TGT_DIR="/srv/repository/main/openapi"
LAST_SPEC="$( ls -lt ${TGT_DIR}/stable/ | grep 'jellyfin-openapi' | head -1 | awk '{ print $NF }' )"
# If new and previous spec don't differ (diff retcode 0), remove incoming and finish
if diff /srv/incoming/openapi/stable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}/openapi.json ${TGT_DIR}/stable/${LAST_SPEC} &>/dev/null; then
rm -r /srv/incoming/openapi/stable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}
exit 0
fi
# Move new spec into place
sudo mv /srv/incoming/openapi/stable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}/openapi.json ${TGT_DIR}/stable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}.json
# Delete previous jellyfin-openapi-stable_previous.json
sudo rm ${TGT_DIR}/jellyfin-openapi-stable_previous.json
# Move current jellyfin-openapi-stable.json symlink to jellyfin-openapi-stable_previous.json
sudo mv ${TGT_DIR}/jellyfin-openapi-stable.json ${TGT_DIR}/jellyfin-openapi-stable_previous.json
# Create new jellyfin-openapi-stable.json symlink
sudo ln -s stable/jellyfin-openapi-${{ env.JELLYFIN_VERSION }}.json ${TGT_DIR}/jellyfin-openapi-stable.json
# Check that the previous openapi stable spec link is correct
if [[ "$( readlink ${TGT_DIR}/jellyfin-openapi-stable_previous.json )" != "stable/${LAST_SPEC}" ]]; then
sudo rm ${TGT_DIR}/jellyfin-openapi-stable_previous.json
sudo ln -s stable/${LAST_SPEC} ${TGT_DIR}/jellyfin-openapi-stable_previous.json
fi
) 200>/run/workflows/openapi-stable.lock
@@ -1,80 +0,0 @@
name: OpenAPI Check
on:
pull_request:
jobs:
ancestor:
name: Common Ancestor
runs-on: ubuntu-latest
outputs:
base_ref: ${{ steps.ancestor.outputs.base_ref }}
steps:
- name: Checkout Repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
fetch-depth: 0
- name: Search History
id: ancestor
run: |
git remote add upstream https://github.com/${{ github.event.pull_request.base.repo.full_name }}
git fetch --prune --progress --no-recurse-submodules upstream +refs/heads/*:refs/remotes/upstream/* +refs/tags/*:refs/tags/*
ANCESTOR_REF=$(git merge-base upstream/${{ github.base_ref }} HEAD)
echo "ref: ${ANCESTOR_REF}"
echo "base_ref=${ANCESTOR_REF}" >> "$GITHUB_OUTPUT"
head:
name: Head Artifact
uses: ./.github/workflows/openapi-generate.yml
with:
ref: ${{ github.event.pull_request.head.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name }}
artifact: openapi-head
base:
name: Base Artifact
uses: ./.github/workflows/openapi-generate.yml
needs:
- ancestor
with:
ref: ${{ needs.ancestor.outputs.base_ref }}
repository: ${{ github.event.pull_request.base.repo.full_name }}
artifact: openapi-base
diff:
name: Generate Report
runs-on: ubuntu-latest
needs:
- head
- base
steps:
- name: Download Head
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: openapi-head
path: openapi-head
- name: Download Base
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: openapi-base
path: openapi-base
- name: Detect Changes
id: openapi-diff
run: |
sed -i 's:allOf:oneOf:g' openapi-head/openapi.json
sed -i 's:allOf:oneOf:g' openapi-base/openapi.json
mkdir -p /tmp/openapi-report
mv openapi-head/openapi.json /tmp/openapi-report/head.json
mv openapi-base/openapi.json /tmp/openapi-report/base.json
docker run -v /tmp/openapi-report:/data openapitools/openapi-diff:2.1.6 /data/base.json /data/head.json --state -l ERROR --markdown /data/openapi-report.md
- name: Upload Artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: openapi-report
path: /tmp/openapi-report/openapi-report.md
@@ -1,59 +0,0 @@
name: OpenAPI Report
on:
workflow_run:
workflows:
- OpenAPI Check
types:
- completed
jobs:
metadata:
name: Generate Metadata
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
outputs:
pr_number: ${{ steps.pr_number.outputs.pr_number }}
steps:
- name: Get Pull Request Number
id: pr_number
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
run: |
API_RESPONSE=$(gh pr list --repo "${GITHUB_REPOSITORY}" --search "${HEAD_SHA}" --state open --json number)
PR_NUMBER=$(echo "${API_RESPONSE}" | jq '.[0].number')
echo "repository: ${GITHUB_REPOSITORY}"
echo "sha: ${HEAD_SHA}"
echo "response: ${API_RESPONSE}"
echo "pr: ${PR_NUMBER}"
echo "pr_number=${PR_NUMBER}" >> "${GITHUB_OUTPUT}"
comment:
name: Pull Request Comment
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
needs:
- metadata
permissions:
pull-requests: write
actions: read
contents: read
steps:
- name: Download OpenAPI Report
id: download_report
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: openapi-report
path: openapi-report
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Push Comment
uses: thollander/actions-comment-pull-request@24bffb9b452ba05a4f3f77933840a6a841d1b32b # v3.0.1
with:
github-token: ${{ secrets.JF_BOT_TOKEN }}
file-path: ${{ steps.download_report.outputs.download-path }}/openapi-report.md
pr-number: ${{ needs.metadata.outputs.pr_number }}
comment-tag: openapi-report
-66
View File
@@ -1,66 +0,0 @@
name: Project Automation
on:
push:
branches:
- master
pull_request:
issue_comment:
permissions: {}
jobs:
project:
name: Project board
runs-on: ubuntu-latest
if: ${{ github.repository == 'jellyfin/jellyfin' }}
steps:
- name: Remove from 'Current Release' project
uses: alex-page/github-project-automation-plus@303f24a24c67ce7adf565a07e96720faf126fe36 # v0.9.0
if: (github.event.pull_request || github.event.issue.pull_request) && !contains(github.event.*.labels.*.name, 'stable backport')
continue-on-error: true
with:
project: Current Release
action: delete
column: In progress
repo-token: ${{ secrets.JF_BOT_TOKEN }}
- name: Add to 'Release Next' project
uses: alex-page/github-project-automation-plus@303f24a24c67ce7adf565a07e96720faf126fe36 # v0.9.0
if: (github.event.pull_request || github.event.issue.pull_request) && github.event.action == 'opened'
continue-on-error: true
with:
project: Release Next
column: In progress
repo-token: ${{ secrets.JF_BOT_TOKEN }}
- name: Add to 'Current Release' project
uses: alex-page/github-project-automation-plus@303f24a24c67ce7adf565a07e96720faf126fe36 # v0.9.0
if: (github.event.pull_request || github.event.issue.pull_request) && !contains(github.event.*.labels.*.name, 'stable backport')
continue-on-error: true
with:
project: Current Release
column: In progress
repo-token: ${{ secrets.JF_BOT_TOKEN }}
- name: Check number of comments from the team member
if: github.event.issue.pull_request == '' && github.event.comment.author_association == 'MEMBER'
id: member_comments
run: echo "::set-output name=number::$(curl -s ${{ github.event.issue.comments_url }} | jq '.[] | select(.author_association == "MEMBER") | .author_association' | wc -l)"
- name: Move issue to needs triage
uses: alex-page/github-project-automation-plus@303f24a24c67ce7adf565a07e96720faf126fe36 # v0.9.0
if: github.event.issue.pull_request == '' && github.event.comment.author_association == 'MEMBER' && steps.member_comments.outputs.number <= 1
continue-on-error: true
with:
project: Issue Triage for Main Repo
column: Needs triage
repo-token: ${{ secrets.JF_BOT_TOKEN }}
- name: Add issue to triage project
uses: alex-page/github-project-automation-plus@303f24a24c67ce7adf565a07e96720faf126fe36 # v0.9.0
if: github.event.issue.pull_request == '' && github.event.action == 'opened'
continue-on-error: true
with:
project: Issue Triage for Main Repo
column: Pending response
repo-token: ${{ secrets.JF_BOT_TOKEN }}
@@ -1,24 +0,0 @@
name: Merge Conflict Labeler
on:
push:
branches:
- master
pull_request_target:
types: [synchronize]
permissions: {}
jobs:
main:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
if: ${{ github.repository == 'jellyfin/jellyfin' }}
steps:
- name: Apply label
uses: eps1lon/actions-label-merge-conflict@0273be72a0bbd58fcd71d0d6c02c209b50d1e5e1 # v3.1.0
with:
dirtyLabel: 'merge conflict'
commentOnDirty: 'This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.'
repoToken: ${{ secrets.JF_BOT_TOKEN }}
-30
View File
@@ -1,30 +0,0 @@
name: Stale PR Check
on:
schedule:
- cron: '30 */12 * * *'
workflow_dispatch:
permissions:
pull-requests: write
actions: write
jobs:
prs-stale-conflicts:
name: Check PRs with merge conflicts
runs-on: ubuntu-latest
if: ${{ contains(github.repository, 'jellyfin/') }}
steps:
- uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
with:
repo-token: ${{ secrets.JF_BOT_TOKEN }}
ascending: true
operations-per-run: 150
# The merge conflict action will remove the label when updated
remove-stale-when-updated: false
days-before-stale: -1
days-before-close: 90
days-before-issue-close: -1
stale-pr-label: merge conflict
close-pr-message: |-
This PR has been closed due to having unresolved merge conflicts.
@@ -1,82 +0,0 @@
name: '🆙 Auto bump_version'
on:
release:
types:
- published
workflow_dispatch:
inputs:
TAG_BRANCH:
required: true
description: release-x.y.z
NEXT_VERSION:
required: true
description: x.y.z
jobs:
auto_bump_version:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'release' && !contains(github.event.release.tag_name, 'rc') }}
env:
TAG_BRANCH: ${{ github.event.release.target_commitish }}
steps:
- name: Wait for deploy checks to finish
uses: jitterbit/await-check-suites@292a541bb7618078395b2ce711a0d89cfb8a568a # v1
with:
ref: ${{ env.TAG_BRANCH }}
intervalSeconds: 60
timeoutSeconds: 3600
- name: Setup YQ
uses: chrisdickinson/setup-yq@fa3192edd79d6eb0e4e12de8dde3a0c26f2b853b # latest
with:
yq-version: v4.9.8
- name: Checkout Repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ env.TAG_BRANCH }}
- name: Setup EnvVars
run: |-
CURRENT_VERSION=$(yq e '.version' build.yaml)
CURRENT_MAJOR_MINOR=${CURRENT_VERSION%.*}
CURRENT_PATCH=${CURRENT_VERSION##*.}
echo "CURRENT_VERSION=${CURRENT_VERSION}" >> $GITHUB_ENV
echo "CURRENT_MAJOR_MINOR=${CURRENT_MAJOR_MINOR}" >> $GITHUB_ENV
echo "CURRENT_PATCH=${CURRENT_PATCH}" >> $GITHUB_ENV
echo "NEXT_VERSION=${CURRENT_MAJOR_MINOR}.$(($CURRENT_PATCH + 1))" >> $GITHUB_ENV
- name: Run bump_version
run: ./bump_version ${{ env.NEXT_VERSION }}
- name: Commit Changes
run: |-
git config user.name "jellyfin-bot"
git config user.email "team@jellyfin.org"
git checkout ${{ env.TAG_BRANCH }}
git commit -am "Bump version to ${{ env.NEXT_VERSION }}"
git push origin ${{ env.TAG_BRANCH }}
manual_bump_version:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'workflow_dispatch' }}
env:
TAG_BRANCH: ${{ github.event.inputs.TAG_BRANCH }}
NEXT_VERSION: ${{ github.event.inputs.NEXT_VERSION }}
steps:
- name: Checkout Repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ env.TAG_BRANCH }}
- name: Run bump_version
run: ./bump_version ${{ env.NEXT_VERSION }}
- name: Commit Changes
run: |-
git config user.name "jellyfin-bot"
git config user.email "team@jellyfin.org"
git checkout ${{ env.TAG_BRANCH }}
git commit -am "Bump version to ${{ env.NEXT_VERSION }}"
git push origin ${{ env.TAG_BRANCH }}
+79
View File
@@ -0,0 +1,79 @@
when:
- event: pull_request
- event: push
steps:
# Restore, build and test in one step so the NuGet cache stays on the step's
# own ephemeral storage instead of the 10Gi workspace volume.
# global.json pins the .NET 10 SDK (10.0.0, rollForward latestMinor).
- name: build-test
image: mcr.microsoft.com/dotnet/sdk:10.0
environment:
DOTNET_CLI_TELEMETRY_OPTOUT: "1"
DOTNET_NOLOGO: "1"
commands:
- dotnet --info
- dotnet restore Jellyfin.sln
- dotnet build Jellyfin.sln -c Release --no-restore
# libSkiaSharp needs fontconfig to load and the SDK image does not ship it.
# The mirror occasionally serves a half-synced index, so update is retried.
- apt-get -o Acquire::Retries=3 update || apt-get -o Acquire::Retries=3 update
- apt-get install -y --no-install-recommends libfontconfig1
- ldconfig -p | grep -q libfontconfig
# BackupServiceTests is excluded because BackupService refuses to write a
# backup with less than 5GiB free on its target path. The clone and build
# output fill 5.7G of the 9.8G workspace volume, leaving 4.1G. Raising the
# agent's workspace volume size is the only way to run it here.
- df -h .
- dotnet test Jellyfin.sln -c Release --no-build --verbosity minimal --filter "Category!=RequiresDocker&FullyQualifiedName!~Integration&FullyQualifiedName!~BackupServiceTests"
backend_options:
kubernetes:
serviceAccountName: jellyfin-ha-src
resources:
requests:
memory: 2Gi
cpu: 2
ephemeral-storage: 10Gi
limits:
memory: 8Gi
cpu: 4
ephemeral-storage: 20Gi
# The PostgreSQL migration tests are the only ones that run the startup migration chain against
# the provider production uses, and the filter above has always skipped them.
# The server runs inside this step: the kubernetes backend has no docker daemon for
# testcontainers, and a postgres service container deadlocks the step because the backend mounts
# the ReadWriteOnce workspace volume into service pods and schedules them on another node.
# Its data directory lives on the step's ephemeral storage, not on the workspace volume.
# Scoped to Jellyfin.Server.Tests: the three classes in Jellyfin.Database.Tests.PostgreSQL still
# start their own container, and PostgreSqlProviderTests already fails on main - on an EF 10
# scalar query and on its own data - which a third test in the class then inherits.
- name: postgres-migration-chain
image: mcr.microsoft.com/dotnet/sdk:10.0
depends_on:
- build-test
environment:
DOTNET_CLI_TELEMETRY_OPTOUT: "1"
DOTNET_NOLOGO: "1"
JELLYFIN_TEST_POSTGRES: "Host=127.0.0.1;Port=5432;Database=postgres;Username=postgres"
commands:
- apt-get -o Acquire::Retries=3 update || apt-get -o Acquire::Retries=3 update
- DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends postgresql
- install -d -o postgres -g postgres /tmp/pgdata /tmp/pgrun
- PGBIN=$(ls -d /usr/lib/postgresql/*/bin | tail -1)
- su postgres -c "$PGBIN/initdb -D /tmp/pgdata -A trust -U postgres"
- su postgres -c "$PGBIN/pg_ctl -D /tmp/pgdata -o \"-c listen_addresses=127.0.0.1 -k /tmp/pgrun\" -l /tmp/pg.log -w start"
- dotnet build tests/Jellyfin.Server.Tests/Jellyfin.Server.Tests.csproj -c Release
- dotnet test tests/Jellyfin.Server.Tests/Jellyfin.Server.Tests.csproj -c Release --no-build --verbosity minimal --filter "Category=RequiresDocker"
backend_options:
kubernetes:
serviceAccountName: jellyfin-ha-src
resources:
requests:
memory: 2Gi
cpu: 2
ephemeral-storage: 6Gi
limits:
memory: 6Gi
cpu: 4
ephemeral-storage: 12Gi
+1
View File
@@ -240,6 +240,7 @@
- [Florin-Popescu](https://github.com/Florin-Popescu)
- [m0g3r](https://github.com/m0g3r)
- [martin-77](https://github.com/martin-77)
- [Oggeb1](https://github.com/Oggeb1)
# Emby Contributors
+8 -1
View File
@@ -8,8 +8,9 @@
<PackageVersion Include="AutoFixture.AutoMoq" Version="4.18.1" />
<PackageVersion Include="AutoFixture.Xunit3" Version="4.19.0" />
<PackageVersion Include="AutoFixture" Version="4.18.1" />
<PackageVersion Include="AWSSDK.S3" Version="4.0.103.1" />
<PackageVersion Include="BDInfo" Version="0.8.0" />
<PackageVersion Include="BitFaster.Caching" Version="2.6.0" />
<PackageVersion Include="BitFaster.Caching" Version="2.6.1" />
<PackageVersion Include="BlurHashSharp.SkiaSharp" Version="1.4.0-pre.1" />
<PackageVersion Include="BlurHashSharp" Version="1.4.0-pre.1" />
<PackageVersion Include="CommandLineParser" Version="2.9.1" />
@@ -46,6 +47,7 @@
<PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.11" />
<PackageVersion Include="Microsoft.Extensions.Http" Version="10.0.11" />
<PackageVersion Include="Microsoft.Extensions.Logging" Version="10.0.11" />
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.11" />
<PackageVersion Include="Microsoft.Extensions.Options" Version="10.0.11" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageVersion Include="MimeTypes" Version="2.5.2" />
@@ -53,6 +55,8 @@
<PackageVersion Include="Moq" Version="4.18.4" />
<PackageVersion Include="NEbml" Version="1.1.0.5" />
<PackageVersion Include="Newtonsoft.Json" Version="13.0.4" />
<PackageVersion Include="Npgsql" Version="10.0.3" />
<PackageVersion Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="10.0.3" />
<PackageVersion Include="PDFtoImage" Version="5.2.1" />
<PackageVersion Include="PlaylistsNET" Version="1.4.1" />
<PackageVersion Include="prometheus-net.AspNetCore" Version="8.2.1" />
@@ -74,12 +78,15 @@
<PackageVersion Include="SkiaSharp.HarfBuzz" Version="3.119.4" />
<PackageVersion Include="SkiaSharp.NativeAssets.Linux" Version="3.119.4" />
<PackageVersion Include="SmartAnalyzers.MultithreadingAnalyzer" Version="1.1.31" />
<PackageVersion Include="StackExchange.Redis" Version="2.13.17" />
<PackageVersion Include="StyleCop.Analyzers" Version="1.2.0-beta.556" />
<PackageVersion Include="Svg.Skia" Version="3.7.0" />
<PackageVersion Include="Swashbuckle.AspNetCore.ReDoc" Version="10.2.3" />
<PackageVersion Include="Swashbuckle.AspNetCore" Version="10.2.3" />
<PackageVersion Include="System.Text.Json" Version="10.0.11" />
<PackageVersion Include="TagLibSharp" Version="2.3.0" />
<PackageVersion Include="Testcontainers.PostgreSql" Version="4.15.0" />
<PackageVersion Include="Testcontainers.Redis" Version="4.15.0" />
<PackageVersion Include="z440.atl.core" Version="7.16.0" />
<PackageVersion Include="TMDbLib" Version="3.0.0" />
<PackageVersion Include="UTF.Unknown" Version="2.7.0" />
+96
View File
@@ -0,0 +1,96 @@
# syntax=docker/dockerfile:1
# ── Build stage ──────────────────────────────────────────────────────────────
FROM --platform=linux/amd64 mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
# Restore dependencies first (layer-cache friendly)
COPY ["Jellyfin.sln", "global.json", "nuget.config", "Directory.Build.props", "Directory.Packages.props", "./"]
COPY ["SharedVersion.cs", "BannedSymbols.txt", "stylecop.json", "./"]
# Copy all project files so dotnet restore can resolve the full dependency graph
COPY Emby.Naming/ Emby.Naming/
COPY Emby.Photos/ Emby.Photos/
COPY Emby.Server.Implementations/ Emby.Server.Implementations/
COPY Jellyfin.Api/ Jellyfin.Api/
COPY Jellyfin.Data/ Jellyfin.Data/
COPY Jellyfin.Server/ Jellyfin.Server/
COPY Jellyfin.Server.Implementations/ Jellyfin.Server.Implementations/
COPY MediaBrowser.Common/ MediaBrowser.Common/
COPY MediaBrowser.Controller/ MediaBrowser.Controller/
COPY MediaBrowser.LocalMetadata/ MediaBrowser.LocalMetadata/
COPY MediaBrowser.MediaEncoding/ MediaBrowser.MediaEncoding/
COPY MediaBrowser.Model/ MediaBrowser.Model/
COPY MediaBrowser.Providers/ MediaBrowser.Providers/
COPY MediaBrowser.XbmcMetadata/ MediaBrowser.XbmcMetadata/
COPY src/ src/
RUN dotnet restore Jellyfin.Server/Jellyfin.Server.csproj \
--runtime linux-x64
# Publish the server (and all transitive dependencies, including the
# PostgreSQL provider assembly added by this fork).
# Note: TreatWarningsAsErrors is disabled for the Docker build — StyleCop
# analyzer violations in upstream src/ projects would otherwise block the
# image build. StyleCop is enforced in the CI pipeline, not the Dockerfile.
RUN dotnet publish Jellyfin.Server/Jellyfin.Server.csproj \
--configuration Release \
--runtime linux-x64 \
--self-contained false \
--no-restore \
-p:TreatWarningsAsErrors=false \
--output /app
# ── Web client stage ──────────────────────────────────────────────────────────
# Install jellyfin-web via the official Jellyfin apt repo.
# Package suffix in the bookworm repo is +deb12 (e.g. 12.0+deb12).
# Web assets land at /usr/share/jellyfin/web/ — stable, prebuilt, no npm required.
# The web client version must match the server version this fork is based on.
FROM --platform=linux/amd64 debian:bookworm-slim AS webclient
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl gnupg ca-certificates \
&& curl -fsSL https://repo.jellyfin.org/jellyfin_team.gpg.key \
| gpg --dearmor -o /usr/share/keyrings/jellyfin.gpg \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/jellyfin.gpg] https://repo.jellyfin.org/debian bookworm main" \
> /etc/apt/sources.list.d/jellyfin.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends "jellyfin-web=12.0+deb12" \
&& rm -rf /var/lib/apt/lists/* \
&& echo "Web client files:" && ls /usr/share/jellyfin/web/ | head -10
# ── Runtime stage ─────────────────────────────────────────────────────────────
FROM --platform=linux/amd64 mcr.microsoft.com/dotnet/aspnet:10.0
# Install FFmpeg and native dependencies required by SkiaSharp and fontconfig.
# libicu, libssl, and liblttng-ust are already present in the dotnet/aspnet base image.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ffmpeg \
fontconfig \
libfontconfig1 \
libfreetype6 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /jellyfin
COPY --from=build /app .
COPY --from=webclient /usr/share/jellyfin/web ./jellyfin-web/
# Jellyfin default ports
EXPOSE 8096
EXPOSE 8920
# Data / config volumes
VOLUME ["/config", "/cache", "/media"]
ENV JELLYFIN_DATA_DIR=/config \
JELLYFIN_CACHE_DIR=/cache \
JELLYFIN_LOG_DIR=/config/log \
JELLYFIN_CONFIG_DIR=/config
ENTRYPOINT ["./jellyfin", \
"--datadir", "/config", \
"--cachedir", "/cache", \
"--webdir", "/jellyfin/jellyfin-web"]
+55
View File
@@ -0,0 +1,55 @@
# syntax=docker/dockerfile:1
# Runtime-only image — the .NET publish step runs on the CI host (runner),
# not inside this Dockerfile.
# ── Web client stage ──────────────────────────────────────────────────────────
# Install jellyfin-web via the official Jellyfin apt repo.
# Package suffix in the bookworm repo is +deb12 (e.g. 12.0+deb12).
# Web assets land at /usr/share/jellyfin/web/ — stable, prebuilt, no npm required.
# The web client version must match the server version this fork is based on.
FROM --platform=linux/amd64 debian:bookworm-slim AS webclient
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl gnupg ca-certificates \
&& curl -fsSL https://repo.jellyfin.org/jellyfin_team.gpg.key \
| gpg --dearmor -o /usr/share/keyrings/jellyfin.gpg \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/jellyfin.gpg] https://repo.jellyfin.org/debian bookworm main" \
> /etc/apt/sources.list.d/jellyfin.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends "jellyfin-web=12.0+deb12" \
&& rm -rf /var/lib/apt/lists/*
# ── Runtime stage ─────────────────────────────────────────────────────────────
FROM --platform=linux/amd64 mcr.microsoft.com/dotnet/aspnet:10.0
# Install FFmpeg and native dependencies required by SkiaSharp and fontconfig.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ffmpeg \
fontconfig \
libfontconfig1 \
libfreetype6 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /jellyfin
# Copy the pre-built publish output produced by `dotnet publish` on the CI host.
COPY publish-output/ .
# Copy the jellyfin-web client assets from the webclient stage.
COPY --from=webclient /usr/share/jellyfin/web ./jellyfin-web/
# Jellyfin default ports
EXPOSE 8096
EXPOSE 8920
# Data / config volumes
VOLUME ["/config", "/cache", "/media"]
ENV JELLYFIN_DATA_DIR=/config \
JELLYFIN_CACHE_DIR=/cache \
JELLYFIN_LOG_DIR=/config/log
ENTRYPOINT ["./jellyfin", \
"--datadir", "/config", \
"--cachedir", "/cache", \
"--webdir", "/jellyfin/jellyfin-web"]
@@ -44,7 +44,14 @@ namespace Emby.Naming.ExternalFiles
}
var extension = Path.GetExtension(path.AsSpan());
if (!(_type == DlnaProfileType.Subtitle && _namingOptions.SubtitleFileExtensions.Contains(extension, StringComparison.OrdinalIgnoreCase))
// .idx carries VobSub per-track language metadata. Recognize it here rather
// than adding it to NamingOptions.SubtitleFileExtensions, which also gates
// subtitle uploads/saves.
var isVobSubIndex = _type == DlnaProfileType.Subtitle && extension.Equals(".idx", StringComparison.OrdinalIgnoreCase);
if (!isVobSubIndex
&& !(_type == DlnaProfileType.Subtitle && _namingOptions.SubtitleFileExtensions.Contains(extension, StringComparison.OrdinalIgnoreCase))
&& !(_type == DlnaProfileType.Audio && _namingOptions.AudioFileExtensions.Contains(extension, StringComparison.OrdinalIgnoreCase))
&& !(_type == DlnaProfileType.Lyric && _namingOptions.LyricFileExtensions.Contains(extension, StringComparison.OrdinalIgnoreCase)))
{
+4 -2
View File
@@ -158,7 +158,9 @@ namespace Emby.Naming.TV
if (nextIndex >= name.Length
|| !"0123456789iIpP".Contains(name[nextIndex], StringComparison.Ordinal))
{
if (int.TryParse(endingNumberGroup.ValueSpan, NumberStyles.Integer, CultureInfo.InvariantCulture, out num))
// A range cannot end before it starts, so a lower number belongs to the episode title rather than to a range.
if (int.TryParse(endingNumberGroup.ValueSpan, NumberStyles.Integer, CultureInfo.InvariantCulture, out num)
&& num >= result.EpisodeNumber)
{
result.EndingEpisodeNumber = num;
}
@@ -226,7 +228,7 @@ namespace Emby.Naming.TV
info.SeriesName = result.SeriesName;
}
if (!info.EndingEpisodeNumber.HasValue && info.EpisodeNumber.HasValue)
if (!info.EndingEpisodeNumber.HasValue && result.EndingEpisodeNumber >= info.EpisodeNumber)
{
info.EndingEpisodeNumber = result.EndingEpisodeNumber;
}
+32 -3
View File
@@ -185,6 +185,13 @@ namespace Emby.Server.Implementations.Dto
allCollectionFolders = _libraryManager.GetUserRootFolder().Children.OfType<Folder>().ToList();
}
// Batch-fetch by-name item counts to avoid N+1 queries
Dictionary<Guid, ItemCounts>? itemCountsBatch = null;
if (options.ContainsField(ItemFields.ItemCounts))
{
itemCountsBatch = GetItemCountsBatch(accessibleItems, user);
}
// Batch-fetch child counts for all folders to avoid N+1 queries
Dictionary<Guid, int>? childCountBatch = null;
if (options.ContainsField(ItemFields.ChildCount))
@@ -293,7 +300,7 @@ namespace Emby.Server.Implementations.Dto
if (options.ContainsField(ItemFields.ItemCounts))
{
SetItemByNameInfo(dto, user);
SetItemByNameInfo(dto, user, itemCountsBatch);
}
returnItems[index] = dto;
@@ -518,14 +525,36 @@ namespace Emby.Server.Implementations.Dto
return dto;
}
private void SetItemByNameInfo(BaseItemDto dto, User? user)
private Dictionary<Guid, ItemCounts> GetItemCountsBatch(IReadOnlyList<BaseItem> items, User? user)
{
var result = new Dictionary<Guid, ItemCounts>();
foreach (var group in items.GroupBy(item => item.GetBaseItemKind()))
{
if (!_relatedItemKinds.TryGetValue(group.Key, out var relatedItemKinds))
{
continue;
}
var ids = group.Select(item => item.Id).ToArray();
foreach (var (id, counts) in _libraryManager.GetItemCountsForNameItems(group.Key, ids, relatedItemKinds, user))
{
result[id] = counts;
}
}
return result;
}
private void SetItemByNameInfo(BaseItemDto dto, User? user, IReadOnlyDictionary<Guid, ItemCounts>? prefetchedCounts = null)
{
if (!_relatedItemKinds.TryGetValue(dto.Type, out var relatedItemKinds))
{
return;
}
var counts = _libraryManager.GetItemCountsForNameItem(dto.Type, dto.Id, relatedItemKinds, user);
var counts = prefetchedCounts?.GetValueOrDefault(dto.Id)
?? _libraryManager.GetItemCountsForNameItem(dto.Type, dto.Id, relatedItemKinds, user);
dto.AlbumCount = counts.AlbumCount;
dto.ArtistCount = counts.ArtistCount;
@@ -65,6 +65,7 @@
<ItemGroup>
<PackageReference Include="Ignore" />
<PackageReference Include="StackExchange.Redis" />
</ItemGroup>
<ItemGroup>
@@ -27,6 +27,11 @@ namespace Emby.Server.Implementations.EntryPoints;
/// </summary>
public sealed class LibraryChangedNotifier : IHostedService, IDisposable
{
// A batch holds a live reference to every item it names, so it has to stay small enough that a
// library scan - which changes items faster than any batch window closes - cannot grow it without
// bound. Reached only by a scan; interactive use closes a batch on the window long before this.
internal const int MaxBatchSize = 2000;
private readonly ILibraryManager _libraryManager;
private readonly IServerConfigurationManager _configurationManager;
private readonly IProviderManager _providerManager;
@@ -35,11 +40,11 @@ public sealed class LibraryChangedNotifier : IHostedService, IDisposable
private readonly ILogger<LibraryChangedNotifier> _logger;
private readonly Lock _libraryChangedSyncLock = new();
private readonly List<Folder> _foldersAddedTo = new();
private readonly List<Folder> _foldersRemovedFrom = new();
private readonly List<BaseItem> _itemsAdded = new();
private readonly List<BaseItem> _itemsRemoved = new();
private readonly List<BaseItem> _itemsUpdated = new();
private readonly Dictionary<Guid, Folder> _foldersAddedTo = [];
private readonly Dictionary<Guid, Folder> _foldersRemovedFrom = [];
private readonly Dictionary<Guid, BaseItem> _itemsAdded = [];
private readonly Dictionary<Guid, BaseItem> _itemsRemoved = [];
private readonly Dictionary<Guid, BaseItem> _itemsUpdated = [];
private readonly ConcurrentDictionary<Guid, DateTime> _lastProgressMessageTimes = new();
private Timer? _libraryUpdateTimer;
@@ -173,7 +178,7 @@ public sealed class LibraryChangedNotifier : IHostedService, IDisposable
private void OnLibraryItemRemoved(object? sender, ItemChangeEventArgs e)
=> OnLibraryChange(e.Item, e.Parent, _itemsRemoved, _foldersRemovedFrom);
private void OnLibraryChange(BaseItem item, BaseItem parent, List<BaseItem> itemsList, List<Folder>? foldersList)
private void OnLibraryChange(BaseItem item, BaseItem parent, Dictionary<Guid, BaseItem> itemsList, Dictionary<Guid, Folder>? foldersList)
{
if (!FilterItem(item))
{
@@ -182,23 +187,28 @@ public sealed class LibraryChangedNotifier : IHostedService, IDisposable
lock (_libraryChangedSyncLock)
{
var updateDuration = TimeSpan.FromSeconds(_configurationManager.Configuration.LibraryUpdateDuration);
// The window runs from the first change of a batch and is never extended. Extending it on
// every change would keep a library scan's batch open for the whole scan, and the batch
// holds the items it names alive, so it would grow to the size of the library.
if (_libraryUpdateTimer is null)
{
var updateDuration = TimeSpan.FromSeconds(_configurationManager.Configuration.LibraryUpdateDuration);
_libraryUpdateTimer = new Timer(LibraryUpdateTimerCallback, null, updateDuration, Timeout.InfiniteTimeSpan);
}
else
{
_libraryUpdateTimer.Change(updateDuration, Timeout.InfiniteTimeSpan);
}
if (foldersList is not null && parent is Folder folder)
{
foldersList.Add(folder);
foldersList[folder.Id] = folder;
}
itemsList.Add(item);
itemsList[item.Id] = item;
// A window long enough to cover a burst still has to give way once the batch is large
// enough to be worth sending on its own.
if (_itemsAdded.Count + _itemsRemoved.Count + _itemsUpdated.Count >= MaxBatchSize)
{
_libraryUpdateTimer.Change(TimeSpan.Zero, Timeout.InfiniteTimeSpan);
}
}
}
@@ -211,22 +221,16 @@ public sealed class LibraryChangedNotifier : IHostedService, IDisposable
List<BaseItem> itemsRemoved;
lock (_libraryChangedSyncLock)
{
// Remove dupes in case some were saved multiple times
foldersAddedTo = _foldersAddedTo
.DistinctBy(x => x.Id)
.ToList();
foldersRemovedFrom = _foldersRemovedFrom
.DistinctBy(x => x.Id)
.ToList();
foldersAddedTo = _foldersAddedTo.Values.ToList();
foldersRemovedFrom = _foldersRemovedFrom.Values.ToList();
itemsUpdated = _itemsUpdated
.Where(i => !_itemsAdded.Contains(i))
.DistinctBy(x => x.Id)
.Where(e => !_itemsAdded.ContainsKey(e.Key))
.Select(e => e.Value)
.ToList();
itemsAdded = _itemsAdded.ToList();
itemsRemoved = _itemsRemoved.ToList();
itemsAdded = _itemsAdded.Values.ToList();
itemsRemoved = _itemsRemoved.Values.ToList();
if (_libraryUpdateTimer is not null)
{
@@ -241,6 +245,15 @@ public sealed class LibraryChangedNotifier : IHostedService, IDisposable
_foldersRemovedFrom.Clear();
}
if (itemsAdded.Count == 0
&& itemsUpdated.Count == 0
&& itemsRemoved.Count == 0
&& foldersAddedTo.Count == 0
&& foldersRemovedFrom.Count == 0)
{
return;
}
await SendChangeNotifications(itemsAdded, itemsUpdated, itemsRemoved, foldersAddedTo, foldersRemovedFrom, CancellationToken.None).ConfigureAwait(false);
}
@@ -18,15 +18,17 @@ namespace Emby.Server.Implementations.EntryPoints
public sealed class UserDataChangeNotifier : IHostedService, IDisposable
{
private const int UpdateDuration = 500;
internal const int MaxBatchSize = 2000;
private readonly ISessionManager _sessionManager;
private readonly IUserDataManager _userDataManager;
private readonly IUserManager _userManager;
private readonly Dictionary<Guid, List<BaseItem>> _changedItems = new();
private readonly Dictionary<Guid, Dictionary<Guid, BaseItem>> _changedItems = [];
private readonly Lock _syncLock = new();
private Timer? _updateTimer;
private int _changedItemCount;
/// <summary>
/// Initializes a new instance of the <see cref="UserDataChangeNotifier"/> class.
@@ -69,50 +71,64 @@ namespace Emby.Server.Implementations.EntryPoints
lock (_syncLock)
{
if (_updateTimer is null)
{
_updateTimer = new Timer(
UpdateTimerCallback,
null,
UpdateDuration,
Timeout.Infinite);
}
else
{
_updateTimer.Change(UpdateDuration, Timeout.Infinite);
}
// The window runs from the first change of a batch and is never extended, so a stream
// of changes that never pauses - a library scan - still closes its batches instead of
// holding every item it touched alive until the stream stops.
_updateTimer ??= new Timer(
UpdateTimerCallback,
null,
UpdateDuration,
Timeout.Infinite);
if (!_changedItems.TryGetValue(e.UserId, out List<BaseItem>? keys))
if (!_changedItems.TryGetValue(e.UserId, out Dictionary<Guid, BaseItem>? keys))
{
keys = new List<BaseItem>();
keys = [];
_changedItems[e.UserId] = keys;
}
keys.Add(e.Item);
var baseItem = e.Item;
// Go up one level for indicators
if (baseItem is not null)
{
Track(keys, baseItem);
var parent = baseItem.GetOwner() ?? baseItem.GetParent();
if (parent is not null)
{
keys.Add(parent);
Track(keys, parent);
}
}
// A window long enough to cover a burst still has to give way once the batch is
// large enough to be worth sending on its own.
if (_changedItemCount >= MaxBatchSize)
{
_updateTimer.Change(0, Timeout.Infinite);
}
}
}
private void Track(Dictionary<Guid, BaseItem> keys, BaseItem item)
{
var before = keys.Count;
keys[item.Id] = item;
if (keys.Count != before)
{
_changedItemCount++;
}
}
private async void UpdateTimerCallback(object? state)
{
List<KeyValuePair<Guid, List<BaseItem>>> changes;
List<KeyValuePair<Guid, Dictionary<Guid, BaseItem>>> changes;
lock (_syncLock)
{
// Remove dupes in case some were saved multiple times
changes = _changedItems.ToList();
_changedItems.Clear();
_changedItemCount = 0;
if (_updateTimer is not null)
{
@@ -121,17 +137,22 @@ namespace Emby.Server.Implementations.EntryPoints
}
}
if (changes.Count == 0)
{
return;
}
foreach (var (userId, changedItems) in changes)
{
await _sessionManager.SendMessageToUserSessions(
[userId],
SessionMessageType.UserDataChanged,
() => GetUserDataChangeInfo(userId, changedItems),
() => GetUserDataChangeInfo(userId, changedItems.Values),
default).ConfigureAwait(false);
}
}
private UserDataChangeInfo GetUserDataChangeInfo(Guid userId, List<BaseItem> changedItems)
private UserDataChangeInfo GetUserDataChangeInfo(Guid userId, IEnumerable<BaseItem> changedItems)
{
var user = _userManager.GetUserById(userId)
?? throw new ArgumentException("Invalid user ID", nameof(userId));
@@ -140,7 +161,6 @@ namespace Emby.Server.Implementations.EntryPoints
{
UserId = userId,
UserDataList = changedItems
.DistinctBy(x => x.Id)
.Select(i =>
{
var dto = _userDataManager.GetUserDataDto(i, user);
@@ -8,6 +8,7 @@ using Emby.Server.Implementations.Library;
using MediaBrowser.Controller.Configuration;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Library;
using MediaBrowser.Controller.Providers;
using MediaBrowser.Model.IO;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
@@ -21,6 +22,7 @@ namespace Emby.Server.Implementations.IO
private readonly ILibraryManager _libraryManager;
private readonly IServerConfigurationManager _configurationManager;
private readonly IFileSystem _fileSystem;
private readonly IDirectoryService _directoryService;
private readonly DotIgnoreIgnoreRule _dotIgnoreIgnoreRule;
/// <summary>
@@ -47,6 +49,7 @@ namespace Emby.Server.Implementations.IO
/// <param name="libraryManager">The library manager.</param>
/// <param name="configurationManager">The configuration manager.</param>
/// <param name="fileSystem">The filesystem.</param>
/// <param name="directoryService">The directory service.</param>
/// <param name="appLifetime">The <see cref="IHostApplicationLifetime"/>.</param>
/// <param name="dotIgnoreIgnoreRule">The .ignore rule handler.</param>
public LibraryMonitor(
@@ -54,6 +57,7 @@ namespace Emby.Server.Implementations.IO
ILibraryManager libraryManager,
IServerConfigurationManager configurationManager,
IFileSystem fileSystem,
IDirectoryService directoryService,
IHostApplicationLifetime appLifetime,
DotIgnoreIgnoreRule dotIgnoreIgnoreRule)
{
@@ -61,6 +65,7 @@ namespace Emby.Server.Implementations.IO
_logger = logger;
_configurationManager = configurationManager;
_fileSystem = fileSystem;
_directoryService = directoryService;
_dotIgnoreIgnoreRule = dotIgnoreIgnoreRule;
appLifetime.ApplicationStarted.Register(Start);
@@ -363,6 +368,8 @@ namespace Emby.Server.Implementations.IO
return;
}
_directoryService.Invalidate(path);
// Ignore certain files, If the parent of an ignored path has a change event, ignore that too
foreach (var i in _tempIgnoredPaths.Keys)
{
@@ -3,6 +3,7 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Linq;
@@ -15,7 +16,6 @@ using Emby.Naming.Common;
using Emby.Naming.TV;
using Emby.Naming.Video;
using Emby.Server.Implementations.Library.Resolvers;
using Emby.Server.Implementations.Library.Validators;
using Emby.Server.Implementations.Playlists;
using Emby.Server.Implementations.ScheduledTasks.Tasks;
using Emby.Server.Implementations.Sorting;
@@ -35,7 +35,6 @@ using MediaBrowser.Controller.Entities.Movies;
using MediaBrowser.Controller.IO;
using MediaBrowser.Controller.Library;
using MediaBrowser.Controller.LiveTv;
using MediaBrowser.Controller.MediaEncoding;
using MediaBrowser.Controller.Persistence;
using MediaBrowser.Controller.Playlists;
using MediaBrowser.Controller.Providers;
@@ -75,7 +74,6 @@ namespace Emby.Server.Implementations.Library
private readonly Lazy<IProviderManager> _providerManagerFactory;
private readonly Lazy<IUserViewManager> _userViewManagerFactory;
private readonly IServerApplicationHost _appHost;
private readonly IMediaEncoder _mediaEncoder;
private readonly IFileSystem _fileSystem;
private readonly IItemRepository _itemRepository;
private readonly IItemPersistenceService _persistenceService;
@@ -88,6 +86,7 @@ namespace Emby.Server.Implementations.Library
private readonly ExtraResolver _extraResolver;
private readonly IPathManager _pathManager;
private readonly ILocalizationManager _localization;
private readonly IDirectoryService _directoryService;
private readonly FastConcurrentLru<Guid, BaseItem> _cache;
private readonly DotIgnoreIgnoreRule _dotIgnoreIgnoreRule;
private readonly IMediaStreamRepository _mediaStreamRepository;
@@ -122,7 +121,6 @@ namespace Emby.Server.Implementations.Library
/// <param name="fileSystem">The file system.</param>
/// <param name="providerManagerFactory">The provider manager.</param>
/// <param name="userViewManagerFactory">The user view manager.</param>
/// <param name="mediaEncoder">The media encoder.</param>
/// <param name="itemRepository">The item repository.</param>
/// <param name="persistenceService">The item persistence service.</param>
/// <param name="nextUpService">The next up service.</param>
@@ -148,7 +146,6 @@ namespace Emby.Server.Implementations.Library
IFileSystem fileSystem,
Lazy<IProviderManager> providerManagerFactory,
Lazy<IUserViewManager> userViewManagerFactory,
IMediaEncoder mediaEncoder,
IItemRepository itemRepository,
IItemPersistenceService persistenceService,
INextUpService nextUpService,
@@ -174,7 +171,6 @@ namespace Emby.Server.Implementations.Library
_fileSystem = fileSystem;
_providerManagerFactory = providerManagerFactory;
_userViewManagerFactory = userViewManagerFactory;
_mediaEncoder = mediaEncoder;
_itemRepository = itemRepository;
_persistenceService = persistenceService;
_nextUpService = nextUpService;
@@ -189,6 +185,7 @@ namespace Emby.Server.Implementations.Library
_pathManager = pathManager;
_dotIgnoreIgnoreRule = dotIgnoreIgnoreRule;
_localization = localization;
_directoryService = directoryService;
_extraResolver = new ExtraResolver(loggerFactory.CreateLogger<ExtraResolver>(), namingOptions, directoryService);
_configurationManager.ConfigurationUpdated += ConfigurationUpdated;
@@ -1209,6 +1206,12 @@ namespace Emby.Server.Implementations.Library
.FirstOrDefault();
}
/// <inheritdoc />
public Guid GetPersonId(string name)
{
return GetItemByNameId<Person>(Person.GetPath(name));
}
/// <inheritdoc />
public Person? GetPerson(string name)
{
@@ -1222,6 +1225,33 @@ namespace Emby.Server.Implementations.Library
return null;
}
/// <inheritdoc />
public Person GetOrCreatePerson(string name)
{
var existing = GetPerson(name);
if (existing is not null)
{
return existing;
}
var path = Person.GetPath(name);
var info = Directory.CreateDirectory(path);
var item = new Person
{
Name = name,
Id = GetItemByNameId<Person>(path),
DateCreated = info.CreationTimeUtc,
DateModified = info.LastWriteTimeUtc,
Path = path
};
item.PresentationUniqueKey = item.CreatePresentationUniqueKey();
CreateItem(item, null);
return item;
}
/// <summary>
/// Gets the studio.
/// </summary>
@@ -1354,15 +1384,6 @@ namespace Emby.Server.Implementations.Library
return GetNewItemIdInternal(path, typeof(T), forceCaseInsensitiveId);
}
/// <inheritdoc />
public Task ValidatePeopleAsync(IProgress<double> progress, CancellationToken cancellationToken)
{
// Ensure the location is available.
Directory.CreateDirectory(_configurationManager.ApplicationPaths.PeoplePath);
return new PeopleValidator(this, _logger, _fileSystem).ValidatePeople(cancellationToken, progress);
}
/// <summary>
/// Reloads the root media folder.
/// </summary>
@@ -1489,6 +1510,10 @@ namespace Emby.Server.Implementations.Library
var numComplete = 0;
var numTasks = tasks.Count;
_logger.LogInformation("Running {TaskCount} post-scan task(s)", numTasks);
var phaseStart = Stopwatch.GetTimestamp();
foreach (var task in tasks)
{
// Prevent access to modified closure
@@ -1506,20 +1531,45 @@ namespace Emby.Server.Implementations.Library
progress.Report(innerPercent);
});
_logger.LogDebug("Running post-scan task {0}", task.GetType().Name);
var taskName = task.GetType().Name;
var taskStart = Stopwatch.GetTimestamp();
_logger.LogInformation(
"Running post-scan task {TaskNumber}/{TaskCount}: {TaskName}",
currentNumComplete + 1,
numTasks,
taskName);
try
{
await task.Run(innerProgress, cancellationToken).ConfigureAwait(false);
var elapsed = Stopwatch.GetElapsedTime(taskStart);
_logger.LogInformation(
"Post-scan task {TaskName} completed after {Minutes} minute(s) and {Seconds} seconds",
taskName,
Math.Truncate(elapsed.TotalMinutes),
elapsed.Seconds);
}
catch (OperationCanceledException)
{
_logger.LogInformation("Post-scan task cancelled: {0}", task.GetType().Name);
var elapsed = Stopwatch.GetElapsedTime(taskStart);
_logger.LogInformation(
"Post-scan task {TaskName} cancelled after {Minutes} minute(s) and {Seconds} seconds",
taskName,
Math.Truncate(elapsed.TotalMinutes),
elapsed.Seconds);
throw;
}
catch (Exception ex)
{
_logger.LogError(ex, "Error running post-scan task");
var elapsed = Stopwatch.GetElapsedTime(taskStart);
_logger.LogError(
ex,
"Post-scan task {TaskName} failed after {Minutes} minute(s) and {Seconds} seconds",
taskName,
Math.Truncate(elapsed.TotalMinutes),
elapsed.Seconds);
}
numComplete++;
@@ -1528,6 +1578,12 @@ namespace Emby.Server.Implementations.Library
progress.Report(percent * 100);
}
var phaseElapsed = Stopwatch.GetElapsedTime(phaseStart);
_logger.LogInformation(
"All post-scan tasks completed after {Minutes} minute(s) and {Seconds} seconds",
Math.Truncate(phaseElapsed.TotalMinutes),
phaseElapsed.Seconds);
_persistenceService.UpdateInheritedValues();
progress.Report(100);
@@ -1745,6 +1801,18 @@ namespace Emby.Server.Implementations.Library
return _countService.GetItemCountsForNameItem(kind, id, relatedItemKinds, query);
}
/// <inheritdoc/>
public Dictionary<Guid, ItemCounts> GetItemCountsForNameItems(BaseItemKind kind, IReadOnlyList<Guid> ids, BaseItemKind[] relatedItemKinds, User? user)
{
var query = new InternalItemsQuery(user);
if (user is not null)
{
AddUserToQuery(query, user);
}
return _countService.GetItemCountsForNameItems(kind, ids, relatedItemKinds, query);
}
public Dictionary<Guid, int> GetChildCountBatch(IReadOnlyList<Guid> parentIds, User? user)
{
return _countService.GetChildCountBatch(parentIds, user);
@@ -3720,6 +3788,10 @@ namespace Emby.Server.Implementations.Library
AddMediaPathInternal(name, path, false);
}
}
// The libraries root was listed before this folder existed, so drop that listing:
// anything still reading it resolves the library set without the new folder.
_directoryService.Invalidate(virtualFolderPath);
}
finally
{
@@ -3746,27 +3818,14 @@ namespace Emby.Server.Implementations.Library
var itemUpdateType = ItemUpdateType.MetadataDownload;
var saveEntity = false;
var createEntity = false;
var personEntity = GetPerson(person.Name);
if (personEntity is null)
{
try
{
var path = Person.GetPath(person.Name);
var info = Directory.CreateDirectory(path);
personEntity = new Person()
{
Name = person.Name,
Id = GetItemByNameId<Person>(path),
DateCreated = info.CreationTimeUtc,
DateModified = info.LastWriteTimeUtc,
Path = path
};
personEntity.PresentationUniqueKey = personEntity.CreatePresentationUniqueKey();
personEntity = GetOrCreatePerson(person.Name);
saveEntity = true;
createEntity = true;
}
catch (Exception ex)
{
@@ -3800,11 +3859,6 @@ namespace Emby.Server.Implementations.Library
if (saveEntity)
{
if (createEntity)
{
CreateItems([personEntity], null, CancellationToken.None);
}
await RunMetadataSavers(personEntity, itemUpdateType).ConfigureAwait(false);
personEntity.DateLastSaved = DateTime.UtcNow;
@@ -3920,6 +3974,7 @@ namespace Emby.Server.Implementations.Library
try
{
Directory.Delete(path, true);
_directoryService.Invalidate(path);
}
finally
{
@@ -3989,6 +4044,7 @@ namespace Emby.Server.Implementations.Library
if (!string.IsNullOrEmpty(shortcut))
{
_fileSystem.DeleteFile(shortcut);
_directoryService.Invalidate(shortcut);
}
var libraryOptions = CollectionFolder.GetLibraryOptions(virtualFolderPath);
@@ -4032,6 +4088,7 @@ namespace Emby.Server.Implementations.Library
}
_fileSystem.CreateShortcut(lnk, _appHost.ReverseVirtualPath(path));
_directoryService.Invalidate(lnk);
RemoveContentTypeOverrides(path);
}
@@ -260,7 +260,7 @@ public sealed class MovieSimilarItemsProvider : ILocalSimilarItemsProvider<Movie
}
var candidateRows = await context.ItemValuesMap.AsNoTracking()
.Where(m => m.ItemValue.Type == valueType && allKeys.Contains(m.ItemValue.CleanValue))
.Where(m => !m.Item.PrimaryVersionId.HasValue && m.ItemValue.Type == valueType && allKeys.Contains(m.ItemValue.CleanValue))
.Select(m => new { m.ItemId, Key = m.ItemValue.CleanValue })
.ToListAsync(cancellationToken).ConfigureAwait(false);
@@ -276,6 +276,7 @@ public sealed class MovieSimilarItemsProvider : ILocalSimilarItemsProvider<Movie
if (personSourceRows.Count > 0)
{
var personCandidateRows = await context.PeopleBaseItemMap.AsNoTracking()
.Where(m => !m.Item.PrimaryVersionId.HasValue)
.Where(m => context.PeopleBaseItemMap
.Where(s => sourceIds.Contains(s.ItemId) && _scoredPersonTypes.Contains(s.People.PersonType))
.Select(s => s.PeopleId)
@@ -1,4 +1,5 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Threading;
@@ -61,6 +62,9 @@ public class ArtistsValidator
var count = names.Count;
var refreshed = 0;
var liveIds = new HashSet<Guid>();
var unresolved = 0;
foreach (var name in names)
{
try
@@ -73,13 +77,20 @@ public class ArtistsValidator
// Fall back to GetArtist if not found (creates new item if needed)
item ??= _libraryManager.GetArtist(name);
var isNew = !existingArtistIds.Contains(item.Id);
var neverRefreshed = item.DateLastRefreshed == default;
if (isNew || neverRefreshed)
// A name with no item is nothing to refresh, and nothing to keep alive either.
if (item is not null)
{
await item.RefreshMetadata(cancellationToken).ConfigureAwait(false);
refreshed++;
liveIds.Add(item.Id);
var isNew = !existingArtistIds.Contains(item.Id);
var neverRefreshed = item.DateLastRefreshed == default;
if (isNew || neverRefreshed)
{
await item.RefreshMetadata(cancellationToken).ConfigureAwait(false);
refreshed++;
}
}
}
catch (OperationCanceledException)
@@ -88,6 +99,7 @@ public class ArtistsValidator
}
catch (Exception ex)
{
unresolved++;
_logger.LogError(ex, "Error refreshing {ArtistName}", name);
}
@@ -101,13 +113,26 @@ public class ArtistsValidator
_logger.LogInformation("Refreshed metadata for {RefreshedCount} new artists out of {TotalCount} total", refreshed, count);
// Every name that threw is a name whose artist is missing from the live set, and deleting against
// a live set with holes in it deletes artists the library still refers to. Leave the sweep to a
// run that got a clean read of them.
if (unresolved > 0)
{
_logger.LogWarning(
"Not removing dead artists: {Count} of {TotalCount} names could not be resolved this run",
unresolved,
count);
progress.Report(100);
return;
}
var deadEntities = _libraryManager.GetItemList(new InternalItemsQuery
{
IncludeItemTypes = [BaseItemKind.MusicArtist],
IsDeadArtist = true,
IsLocked = false
}).Cast<MusicArtist>()
.Where(item => item.IsAccessedByName)
}).OfType<MusicArtist>()
.Where(item => item.IsAccessedByName && !liveIds.Contains(item.Id))
.ToList();
foreach (var item in deadEntities)
@@ -1,12 +1,12 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Jellyfin.Data.Enums;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Library;
using MediaBrowser.Controller.Providers;
using MediaBrowser.Model.IO;
using Microsoft.Extensions.Logging;
namespace Emby.Server.Implementations.Library.Validators;
@@ -17,112 +17,143 @@ namespace Emby.Server.Implementations.Library.Validators;
public class PeopleValidator
{
/// <summary>
/// The _library manager.
/// The library manager.
/// </summary>
private readonly ILibraryManager _libraryManager;
/// <summary>
/// The _logger.
/// The logger.
/// </summary>
private readonly ILogger _logger;
private readonly IFileSystem _fileSystem;
private readonly ILogger<PeopleValidator> _logger;
/// <summary>
/// Initializes a new instance of the <see cref="PeopleValidator" /> class.
/// </summary>
/// <param name="libraryManager">The library manager.</param>
/// <param name="logger">The logger.</param>
/// <param name="fileSystem">The file system.</param>
public PeopleValidator(ILibraryManager libraryManager, ILogger logger, IFileSystem fileSystem)
public PeopleValidator(ILibraryManager libraryManager, ILogger<PeopleValidator> logger)
{
_libraryManager = libraryManager;
_logger = logger;
_fileSystem = fileSystem;
}
/// <summary>
/// Validates the people.
/// </summary>
/// <param name="cancellationToken">The cancellation token.</param>
/// <param name="progress">The progress.</param>
/// <param name="cancellationToken">The cancellation token.</param>
/// <returns>Task.</returns>
public async Task ValidatePeople(CancellationToken cancellationToken, IProgress<double> progress)
public async Task Run(IProgress<double> progress, CancellationToken cancellationToken)
{
// Before the refresh below walks them: a credit no item maps to any more stands for nothing,
// and while it is there the person it names cannot reach the dead-person sweep either.
var numOrphaned = _libraryManager.DeleteOrphanedCredits();
if (numOrphaned > 0)
{
_logger.LogDebug("Deleted {Amount} credits no item maps to", numOrphaned);
_logger.LogInformation("Deleted {Amount} credits no item maps to", numOrphaned);
}
var people = _libraryManager.GetPeopleNames(new InternalPeopleQuery());
var names = _libraryManager.GetPeopleNames(new InternalPeopleQuery());
var existingPersonIds = _libraryManager.GetItemIds(new InternalItemsQuery
{
IncludeItemTypes = [BaseItemKind.Person]
}).ToHashSet();
var (newNames, deadIds) = PartitionCreditsByPersonId(names, _libraryManager.GetPersonId, existingPersonIds);
var numComplete = 0;
var count = names.Count;
var refreshed = 0;
var numPeople = people.Count;
IProgress<double> subProgress = new Progress<double>((val) => progress.Report(val / 2));
_logger.LogDebug("Will refresh {Amount} people", numPeople);
foreach (var person in people)
foreach (var name in names)
{
cancellationToken.ThrowIfCancellationRequested();
try
{
var item = _libraryManager.GetPerson(person);
if (item is null)
var item = _libraryManager.GetOrCreatePerson(name);
var isNew = !existingPersonIds.Contains(item.Id);
var neverRefreshed = item.DateLastRefreshed == default;
if (isNew || neverRefreshed)
{
_logger.LogWarning("Failed to get person: {Name}", person);
continue;
await item.RefreshMetadata(cancellationToken).ConfigureAwait(false);
refreshed++;
}
var options = new MetadataRefreshOptions(new DirectoryService(_fileSystem))
{
ImageRefreshMode = MetadataRefreshMode.ValidationOnly,
MetadataRefreshMode = MetadataRefreshMode.ValidationOnly
};
await item.RefreshMetadata(options, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
// Don't clutter the log
throw;
}
catch (Exception ex)
{
_logger.LogError(ex, "Error validating IBN entry {Person}", person);
_logger.LogError(ex, "Error refreshing {PersonName}", name);
}
// Update progress
numComplete++;
double percent = numComplete;
percent /= numPeople;
percent /= count;
percent *= 100;
subProgress.Report(100 * percent);
progress.Report(percent);
}
var deadEntities = _libraryManager.GetItemList(new InternalItemsQuery
{
IncludeItemTypes = [BaseItemKind.Person],
IsDeadPerson = true,
IsLocked = false
});
_logger.LogInformation(
"Refreshed metadata for {RefreshedCount} people out of {TotalCount} total, {NewCount} of which had no item yet",
refreshed,
count,
newNames.Count);
subProgress = new Progress<double>((val) => progress.Report((val / 2) + 50));
// A person somebody locked is theirs, not ours, however little the library still credits them.
var deadEntities = deadIds
.Select(_libraryManager.GetItemById)
.OfType<Person>()
.Where(item => !item.IsLocked)
.ToList();
var i = 0;
foreach (var item in deadEntities.Chunk(500))
foreach (var item in deadEntities)
{
_libraryManager.DeleteItemsUnsafeFast(item, true);
subProgress.Report(100f / deadEntities.Count * (i++ * 100));
_logger.LogInformation("Deleting dead {ItemType} {ItemId} {ItemName}", item.GetType().Name, item.Id.ToString("N", CultureInfo.InvariantCulture), item.Name);
}
_libraryManager.DeleteItemsUnsafeFast(deadEntities, deleteSourceFiles: true);
progress.Report(100);
}
_logger.LogInformation("People validation complete, deleted {Orphaned} orphaned credits", numOrphaned);
/// <summary>
/// Splits the person items into the ones a credit still calls for and the ones nothing does.
/// </summary>
/// <param name="creditNames">Every name credited on an item, from the people table.</param>
/// <param name="getPersonId">Maps a credit name to the id its person item has.</param>
/// <param name="existingPersonIds">The ids of the person items that exist.</param>
/// <returns>The credits needing an item, and the ids of the items nothing credits.</returns>
internal static (List<string> NewNames, List<Guid> DeadIds) PartitionCreditsByPersonId(
IReadOnlyList<string> creditNames,
Func<string, Guid> getPersonId,
IReadOnlySet<Guid> existingPersonIds)
{
ArgumentNullException.ThrowIfNull(creditNames);
ArgumentNullException.ThrowIfNull(getPersonId);
ArgumentNullException.ThrowIfNull(existingPersonIds);
var newNames = new List<string>();
var liveIds = new HashSet<Guid>();
foreach (var name in creditNames)
{
var personId = getPersonId(name);
// Distinct credit names can normalize onto one id; only the first of them needs an item.
if (liveIds.Add(personId) && !existingPersonIds.Contains(personId))
{
newNames.Add(name);
}
}
var deadIds = existingPersonIds.Where(id => !liveIds.Contains(id)).ToList();
return (newNames, deadIds);
}
}
@@ -112,5 +112,11 @@
"NameExtraInterview": "Інтэрв'ю",
"NameExtraNumbered": "{0} {1}",
"NameExtraScene": "Сцэна",
"NameExtraTrailer": "Трэйлер"
"NameExtraTrailer": "Трэйлер",
"NameExtraBehindTheScenes": "За кулісамі",
"NameExtraClip": "Кліп",
"NameExtraFeaturette": "Кароткаметражка",
"NameExtraSample": "Прыклад",
"NameExtraShort": "Кароткаметражка",
"NameExtraThemeSong": "Тэматычная песня"
}
@@ -106,5 +106,17 @@
"TaskMoveTrickplayImages": "Migracija lokacije slike Trickplay",
"TaskMoveTrickplayImagesDescription": "Premješta postojeće datoteke trik-igara prema postavkama biblioteke.",
"CleanupUserDataTask": "Zadatak čišćenja korisničkih podataka",
"CleanupUserDataTaskDescription": "Čisti sve korisničke podatke (stanje praćenja, status omiljenog itd.) sa medija koji više nije prisutan najmanje 90 dana."
"CleanupUserDataTaskDescription": "Čisti sve korisničke podatke (stanje praćenja, status omiljenog itd.) sa medija koji više nije prisutan najmanje 90 dana.",
"NameExtraBehindTheScenes": "Iza kulisa",
"NameExtraClip": "Isječak",
"NameExtraDeletedScene": "Izbrišana scena",
"NameExtraFeaturette": "Kratki prilog",
"NameExtraInterview": "Intervju",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Uzorak",
"NameExtraScene": "Scena",
"NameExtraShort": "Kratko",
"NameExtraThemeSong": "Tema",
"NameExtraThemeVideo": "Tematski video",
"NameExtraTrailer": "Najava"
}
@@ -108,5 +108,18 @@
"CleanupUserDataTaskDescription": "Καθαρίζει όλα τα δεδομένα χρήστη (κατάσταση παρακολούθησης, κατάσταση αγαπημένων κ.λπ.) από πολυμέσα που δεν υπάρχουν πλέον για τουλάχιστον 90 ημέρες.",
"CleanupUserDataTask": "Εργασία εκκαθάρισης δεδομένων χρήστη",
"LyricDownloadFailureFromForItem": "Αποτυχία λήψης στίχων από {0} για {1}",
"Original": "Πρωτότυπο"
"Original": "Πρωτότυπο",
"NameExtraBehindTheScenes": "Πίσω από τις Σκηνές",
"NameExtraDeletedScene": "Διεγραμμένη Σκηνή",
"NameExtraFeaturette": "Πρόσθετα βίντεο",
"NameExtraInterview": "Συνέντευξη",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Δείγμα",
"NameExtraScene": "Σκηνή",
"NameExtraShort": "Βίντεο μικρού μήκους",
"NameExtraThemeSong": "Θεματικό Τραγούδι",
"NameExtraThemeVideo": "Θεματικό Βίντεο",
"NameExtraTrailer": "τρέιλερ ταινίας",
"NameExtraUnknown": "Πρόσθετα",
"NameExtraClip": "Απόσπασμα"
}
@@ -113,5 +113,12 @@
"NameExtraClip": "Klippi",
"NameExtraDeletedScene": "Poistettu Kohtaus",
"NameExtraFeaturette": "Lyhytelokuva",
"NameExtraInterview": "Haastattelu"
"NameExtraInterview": "Haastattelu",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Näyte",
"NameExtraScene": "Kohtaus",
"NameExtraShort": "Lyhytfilmi",
"NameExtraThemeSong": "Tunnusmusiikki",
"NameExtraThemeVideo": "Tunnusvideo",
"NameExtraTrailer": "Traileri"
}
@@ -7,8 +7,8 @@
"AppDeviceValues": "App: {0}, Eind: {1}",
"Books": "Bøkur",
"ChapterNameValue": "Kapittul {0}",
"Favorites": "Yndis",
"Folders": "Mappur",
"Favorites": "Yndislista",
"Folders": "Skjáttur",
"Forced": "Kravt",
"FailedLoginAttemptWithUserName": "Miseydnað innritanarroynd frá {0}",
"HeaderFavoriteEpisodes": "Yndispartar",
@@ -104,7 +104,7 @@
"NotificationOptionCameraImageUploaded": "Ljósmynd uppsend",
"NameExtraShort": "Stuttfilmur",
"NameExtraThemeSong": "Eyðkennislag",
"NameExtraTrailer": "Forfilmur",
"NameExtraTrailer": "Brellbiti",
"NameExtraInterview": "Samrøða",
"NameExtraBehindTheScenes": "Aftanfyri leiktjøldini",
"NameExtraClip": "Klipp",
@@ -112,14 +112,14 @@
"NameExtraFeaturette": "Stuttur heimildarfilmur",
"TaskAudioNormalization": "Ljóðjavnan",
"TaskAudioNormalizationDescription": "Kannar fílur fyri dátum til ljóðjavnan.",
"NameExtraSample": "Kut",
"NameExtraSample": "Sýnislutur",
"TaskRefreshTrickplayImages": "Framleið Trickplay-myndir",
"TaskRefreshTrickplayImagesDescription": "Framleiðir trickplay-myndir fyri kykmyndir í søvnunm har tað er virkt.",
"TaskMoveTrickplayImages": "Flyt Trickplay-myndagoymslustað",
"TaskMoveTrickplayImagesDescription": "Flytur verandi trickplay-fílur sambært savnsstillingunum.",
"NameExtraThemeVideo": "Eyðkenniskykmynd",
"NameExtraDeletedScene": "Úrtikin mynd",
"NameExtraScene": "Mynd (scena)",
"NameExtraScene": "Mynd",
"NameExtraUnknown": "Eykatilfar",
"Original": "Upprunalig(t/ur)"
}
@@ -108,5 +108,17 @@
"CleanupUserDataTask": "Tasc glantacháin sonraí úsáideora",
"CleanupUserDataTaskDescription": "Glanann sé gach sonraí úsáideora (stádas faire, stádas is fearr leat srl.) ó mheáin nach bhfuil i láthair a thuilleadh ar feadh 90 lá ar a laghad.",
"Original": "Bunaidh",
"LyricDownloadFailureFromForItem": "Theip ar liricí a íoslódáil ó {0} do {1}"
"LyricDownloadFailureFromForItem": "Theip ar liricí a íoslódáil ó {0} do {1}",
"NameExtraBehindTheScenes": "Taobh thiar de na Radhairc",
"NameExtraClip": "Gearrthóg",
"NameExtraDeletedScene": "Radharc Scriosta",
"NameExtraFeaturette": "Mionghné",
"NameExtraInterview": "Agallamh",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Sampla",
"NameExtraScene": "Radharc",
"NameExtraShort": "Gearr",
"NameExtraThemeSong": "Amhrán Téama",
"NameExtraThemeVideo": "Físeán Téama",
"NameExtraTrailer": "Leantóir"
}
@@ -108,5 +108,17 @@
"CleanupUserDataTask": "Zadatak čišćenja korisničkih podataka",
"CleanupUserDataTaskDescription": "Briše sve korisničke podatke (stanje gledanja, status favorita itd.) s medija koji više nisu prisutni najmanje 90 dana.",
"Original": "Original",
"LyricDownloadFailureFromForItem": "Preuzimanje tekstova pjesmi od {0} za {1} nije uspjelo"
"LyricDownloadFailureFromForItem": "Preuzimanje tekstova pjesmi od {0} za {1} nije uspjelo",
"NameExtraBehindTheScenes": "Iza kulisa",
"NameExtraClip": "Klip",
"NameExtraDeletedScene": "Obrisana Scena",
"NameExtraFeaturette": "Promotivni video",
"NameExtraInterview": "Intervju",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Uzorak",
"NameExtraScene": "Scena",
"NameExtraShort": "Kratki film",
"NameExtraThemeSong": "Glavna Pjesma",
"NameExtraThemeVideo": "Tema videa",
"NameExtraTrailer": "Trailer"
}
@@ -108,5 +108,17 @@
"LyricDownloadFailureFromForItem": "Feeler beim Download vun de Songtexter vun {0} fir {1}",
"Original": "Original",
"CleanupUserDataTask": "Aufgab fir Berengege vu Benotzerdaten",
"CleanupUserDataTaskDescription": "Läscht all Benotzerdaten (Ofspillstatus, Favoritestatus, asw.) vu Medien, déi zënter mindestens 90 Deeg net méi besteeënd sinn."
"CleanupUserDataTaskDescription": "Läscht all Benotzerdaten (Ofspillstatus, Favoritestatus, asw.) vu Medien, déi zënter mindestens 90 Deeg net méi besteeënd sinn.",
"NameExtraBehindTheScenes": "Hannert de Kulissen",
"NameExtraClip": "Clip",
"NameExtraDeletedScene": "Geläschte Scène",
"NameExtraFeaturette": "Featurette",
"NameExtraInterview": "Interview",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Beispill",
"NameExtraScene": "Scène",
"NameExtraShort": "Kuerzfilm",
"NameExtraThemeSong": "Theme-Lidd",
"NameExtraThemeVideo": "Theme-Video",
"NameExtraTrailer": "Bande-Annonce"
}
@@ -100,14 +100,14 @@
"TaskAudioNormalization": "Garso normalizavimas",
"TaskAudioNormalizationDescription": "Skenuoja failus, ieškant garso normalizavimo duomenų.",
"TaskExtractMediaSegments": "Medijos segmentų nuskaitymas",
"TaskDownloadMissingLyrics": "Parsisiųsti trūkstamus dainų tekstus",
"TaskDownloadMissingLyrics": "Atsisiųsti trūkstamus dainų tekstus",
"TaskExtractMediaSegmentsDescription": "Ištraukia arba gauna medijos segmentus iš MediaSegment ijungtų įskiepių.",
"TaskMoveTrickplayImages": "Pakeisti Trickplay atvaizdų vietą",
"TaskMoveTrickplayImagesDescription": "Perkelia egzistuojančius Trickplay failus pagal bibliotekos nustatymus.",
"TaskDownloadMissingLyricsDescription": "Parsisiųsti dainų žodžius",
"TaskDownloadMissingLyricsDescription": "Atsisiųsti dainų tekstus",
"CleanupUserDataTask": "Naudotojo duomenų valymo užduotis",
"CleanupUserDataTaskDescription": "Iš medijos, kurios nebėra bent 90 dienų, išvalo visus naudotojo duomenis (žiūrėjimo būseną, mėgstamą būseną ir t. t.).",
"LyricDownloadFailureFromForItem": "Nepavyko atsisiųsti dainos žodžių iš {0}, skirto {1}",
"LyricDownloadFailureFromForItem": "Nepavyko atsisiųsti dainos teksto iš {0}, skirto {1}",
"NameExtraBehindTheScenes": "Užkulisiuose",
"NameExtraClip": "Klipas",
"NameExtraDeletedScene": "Ištrinta scena",
@@ -108,5 +108,17 @@
"CleanupUserDataTask": "Lietotāju datu tīrīšanas uzdevums",
"CleanupUserDataTaskDescription": "Notīra visus lietotāja datus (skatīšanās stāvokļus, favorītu statusi utt.) no medijiem, kas vairs nav pieejami vismaz 90 dienas.",
"Original": "Oriģināls",
"LyricDownloadFailureFromForItem": "Dziesmu vārdi nevarēja tikt lejupielādēti no {0} priekš {1}"
"LyricDownloadFailureFromForItem": "Dziesmu vārdi nevarēja tikt lejupielādēti no {0} priekš {1}",
"NameExtraBehindTheScenes": "Aiz kadra",
"NameExtraClip": "Klips",
"NameExtraDeletedScene": "Izdzēsta aina",
"NameExtraFeaturette": "Īsfilma",
"NameExtraInterview": "Intervija",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Paraugs",
"NameExtraScene": "Aina",
"NameExtraShort": "Īsfilma",
"NameExtraThemeSong": "Motīvu dziesma",
"NameExtraThemeVideo": "Tēmas video",
"NameExtraTrailer": "Treileris"
}
@@ -106,5 +106,15 @@
"TaskMoveTrickplayImagesDescription": "Flytter eksisterende Trickplay-filer i henhold til biblioteksinstillingene.",
"TaskExtractMediaSegmentsDescription": "Trekker ut eller henter mediasegmenter fra plugins som støtter MediaSegment.",
"CleanupUserDataTaskDescription": "Sletter all brukerdata (avspillings-status, favoritter osv.) fra innhold som har vært utilgjengelig i minst 90 dager.",
"CleanupUserDataTask": "Oppgave for opprydding av brukerdata"
"CleanupUserDataTask": "Oppgave for opprydding av brukerdata",
"NameExtraBehindTheScenes": "Bak kulissene",
"NameExtraDeletedScene": "Slettet scene",
"NameExtraFeaturette": "Presentasjonsfilm",
"NameExtraInterview": "Intervju",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Prøve",
"NameExtraScene": "Scene",
"NameExtraThemeSong": "Tema-låt",
"NameExtraThemeVideo": "Tema-video",
"NameExtraTrailer": "Trailer"
}
@@ -111,5 +111,14 @@
"Original": "Original",
"NameExtraBehindTheScenes": "În culise",
"NameExtraClip": "Clip",
"NameExtraDeletedScene": "Scenă ștearsă"
"NameExtraDeletedScene": "Scenă ștearsă",
"NameExtraFeaturette": "Material bonus",
"NameExtraInterview": "Interviu",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Monstră",
"NameExtraScene": "Scenă",
"NameExtraShort": "Scurt",
"NameExtraThemeSong": "Audio de Fundal",
"NameExtraThemeVideo": "Video de Fundal",
"NameExtraTrailer": "Trailer"
}
@@ -91,7 +91,7 @@
"Default": "Predvolené",
"TaskOptimizeDatabaseDescription": "Zmenší databázu a odstráni prázdne miesto. Spustenie tejto úlohy po skenovaní knižnice alebo po iných zmenách zahŕňajúcich úpravy databáze môže zlepšiť výkon.",
"TaskOptimizeDatabase": "Optimalizovať databázu",
"TaskKeyframeExtractorDescription": "Extrahuje kľúčové snímky z video súborov na vytvorenie presnejších HLS zoznamov prehrávania. Táto úloha môže trvať dlhšiu dobu.",
"TaskKeyframeExtractorDescription": "Extrahuje kľúčové snímky z videosúborov na vytvorenie presnejších HLS zoznamov. Táto úloha môže trvať dlhší čas.",
"TaskKeyframeExtractor": "Extraktor kľúčových snímkov",
"External": "Externé",
"HearingImpaired": "Sluchovo postihnutí",
@@ -108,5 +108,17 @@
"CleanupUserDataTask": "Čiščenje uporabniških podatkov",
"CleanupUserDataTaskDescription": "Izbriše vse uporabniške podatke (stanje ogleda, priljubljene itd.) za vsebine, ki že več kot 90 dni niso na voljo.",
"LyricDownloadFailureFromForItem": "Besedila ni bilo mogoče prenesti iz {0} za {1}",
"Original": "Original"
"Original": "Original",
"NameExtraBehindTheScenes": "V zakulisju",
"NameExtraClip": "Klip",
"NameExtraDeletedScene": "Izbrisan prizor",
"NameExtraFeaturette": "Kratek dokumentarec o izdelavi filma",
"NameExtraInterview": "Intervju",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "Vzorec",
"NameExtraScene": "Prizor",
"NameExtraShort": "Kratki film",
"NameExtraThemeSong": "Tematska Pesem",
"NameExtraThemeVideo": "Tematski Video",
"NameExtraTrailer": "Napovednik"
}
@@ -22,20 +22,20 @@
"NewVersionIsAvailable": "เวอร์ชันใหม่ของเซิร์ฟเวอร์ Jellyfin พร้อมให้ดาวน์โหลดแล้ว",
"NameSeasonUnknown": "ไม่ทราบซีซัน",
"NameSeasonNumber": "ซีซัน {0}",
"NameInstallFailed": "การติดตั้ง {0} ล้มเหลว",
"NameInstallFailed": "ติดตั้ง {0} ไม่สำเร็จ",
"MusicVideos": "มิวสิควิดีโอ",
"Music": "ดนตรี",
"Music": "เพลง",
"Movies": "ภาพยนตร์",
"MixedContent": "เนื้อหาผสม",
"Latest": "ล่าสุด",
"LabelRunningTimeValue": "ผ่านไปแล้ว: {0}",
"LabelIpAddressValue": "ที่อยู่ IP: {0}",
"Inherit": "สืบทอด",
"HomeVideos": "โฮมวิดีโอ",
"HeaderNextUp": "ถัดไป",
"HeaderLiveTV": "ทีวีสด",
"HeaderFavoriteShows": "รายการที่ชื่นชอบ",
"HeaderFavoriteEpisodes": "ตอนที่ชื่นชอบ",
"MixedContent": "เนื้อหาหลากหลายประเภท",
"Latest": "มาใหม่ล่าสุด",
"LabelRunningTimeValue": "ความยาว: {0}",
"LabelIpAddressValue": "หมายเลข IP: {0}",
"Inherit": "ใช้ค่าเริ่มต้น",
"HomeVideos": "วิดีโอส่วนตัว",
"HeaderNextUp": "รายการถัดไป",
"HeaderLiveTV": "ทีวีถ่ายทอดสด",
"HeaderFavoriteShows": "รายการที่ชอบ",
"HeaderFavoriteEpisodes": "ตอนที่ชอบ",
"HeaderContinueWatching": "ดูต่อ",
"Genres": "ประเภท",
"Folders": "โฟลเดอร์",
@@ -107,6 +107,19 @@
"TaskMoveTrickplayImages": "ย้ายตำแหน่งเก็บภาพตัวอย่าง Trickplay",
"CleanupUserDataTask": "ส่วนงานล้างข้อมูลผู้ใช้",
"CleanupUserDataTaskDescription": "ล้างข้อมูลผู้ใช้ทั้งหมด (สถานะการรับชม สถานะรายการโปรด ฯลฯ) จากสื่อที่ไม่ได้ใช้งานแล้วอย่างน้อย 90 วัน",
"LyricDownloadFailureFromForItem": "ไม่สามารถดาวน์โหลดเนื้อเพลงจาก {0} สำหรับ {1}",
"Original": "ต้นฉบับ"
"LyricDownloadFailureFromForItem": "ดาวน์โหลดเนื้อเพลงจาก {0} สำหรับ {1} ไม่สำเร็จ",
"Original": "ต้นฉบับ",
"NameExtraBehindTheScenes": "เบื้องหลังการถ่ายทำ",
"NameExtraClip": "คลิปวิดีโอ",
"NameExtraDeletedScene": "ฉากที่ถูกตัดออก",
"NameExtraFeaturette": "คลิปสั้นพิเศษ",
"NameExtraInterview": "บทสัมภาษณ์",
"NameExtraNumbered": "{0} {1}",
"NameExtraSample": "ตัวอย่าง",
"NameExtraScene": "ฉาก",
"NameExtraShort": "ภาพยนตร์สั้น",
"NameExtraThemeSong": "เพลงประกอบ",
"NameExtraThemeVideo": "วิดีโอธีม",
"NameExtraTrailer": "ตัวอย่างภาพยนตร์",
"NameExtraUnknown": "เนื้อหาพิเศษ"
}
@@ -0,0 +1,235 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Controller.MediaEncoding;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StackExchange.Redis;
namespace Emby.Server.Implementations.MediaEncoding;
/// <summary>
/// A Redis-backed implementation of <see cref="ITranscodeSessionStore"/> that provides
/// durable, distributed session tracking with lease-based ownership between pods.
/// </summary>
public sealed class RedisTranscodeSessionStore : ITranscodeSessionStore
{
private const string KeyPrefix = "jellyfin:transcode:";
/// <summary>
/// Lua script for atomic takeover: reads the stored session, checks whether the lease has
/// expired (comparing the stored expiry against the caller-supplied current time) and, if it
/// has, claims it for the calling pod before returning 1; returns 0 otherwise.
/// </summary>
private const string TakeoverScript = @"
local raw = redis.call('GET', KEYS[1])
if not raw then return 0 end
local session = cjson.decode(raw)
if tonumber(session['LeaseExpiresUtc']) > tonumber(ARGV[1]) then return 0 end
session['OwnerPod'] = ARGV[2]
session['LeaseExpiresUtc'] = tonumber(ARGV[1]) + tonumber(ARGV[3])
redis.call('SET', KEYS[1], cjson.encode(session), 'PX', tonumber(ARGV[4]))
return 1";
/// <summary>
/// Lua script for atomic, ownership-checked renewal: extends the lease only while the calling
/// pod still owns it, so a renewal racing a successful takeover cannot revert the new owner.
/// </summary>
private const string RenewScript = @"
local raw = redis.call('GET', KEYS[1])
if not raw then return 0 end
local session = cjson.decode(raw)
if session['OwnerPod'] ~= ARGV[2] then return 0 end
session['LeaseExpiresUtc'] = tonumber(ARGV[1]) + tonumber(ARGV[3])
redis.call('SET', KEYS[1], cjson.encode(session), 'PX', tonumber(ARGV[4]))
return 1";
// The lease expiry is serialized as unix milliseconds because the Lua scripts compare it
// numerically; an ISO-8601 string cannot be compared against a number in Lua.
private static readonly JsonSerializerOptions _jsonOptions = new JsonSerializerOptions
{
Converters = { new UnixMillisecondsDateTimeConverter() }
};
private readonly IConnectionMultiplexer _redis;
private readonly IDatabase _db;
private readonly TranscodeStoreOptions _options;
private readonly ILogger<RedisTranscodeSessionStore> _logger;
/// <summary>
/// Initializes a new instance of the <see cref="RedisTranscodeSessionStore"/> class.
/// </summary>
/// <param name="redis">The Redis connection multiplexer.</param>
/// <param name="options">The transcode store configuration options.</param>
/// <param name="logger">The logger.</param>
public RedisTranscodeSessionStore(
IConnectionMultiplexer redis,
IOptions<TranscodeStoreOptions> options,
ILogger<RedisTranscodeSessionStore> logger)
{
_redis = redis;
_db = redis.GetDatabase();
_options = options.Value;
_logger = logger;
}
private long LeaseDurationMs => (long)_options.LeaseDurationSeconds * 1000;
private long RetentionMs => Math.Max((long)_options.SessionRetentionSeconds * 1000, LeaseDurationMs);
/// <inheritdoc />
public async Task SetAsync(TranscodeSession session, CancellationToken cancellationToken = default)
{
var key = GetKey(session.PlaySessionId);
var json = JsonSerializer.Serialize(session, _jsonOptions);
await _db.StringSetAsync(key, json, TimeSpan.FromMilliseconds(RetentionMs)).ConfigureAwait(false);
_logger.LogDebug("Set transcode session {PlaySessionId} in Redis.", session.PlaySessionId);
}
/// <inheritdoc />
public async Task<TranscodeSession?> TryGetAsync(string playSessionId, CancellationToken cancellationToken = default)
{
var key = GetKey(playSessionId);
var raw = await _db.StringGetAsync(key).ConfigureAwait(false);
if (!raw.HasValue)
{
return null;
}
var session = JsonSerializer.Deserialize<TranscodeSession>(raw.ToString(), _jsonOptions);
// The record outlives the lease so that an orphaned session can still be taken over,
// so the lease has to be checked explicitly here.
if (session is null || session.LeaseExpiresUtc <= DateTime.UtcNow)
{
return null;
}
return session;
}
/// <inheritdoc />
public async Task<bool> RenewLeaseAsync(string playSessionId, string ownerPod, CancellationToken cancellationToken = default)
{
var result = (long?)await _db.ScriptEvaluateAsync(
RenewScript,
keys: new RedisKey[] { GetKey(playSessionId) },
values: new RedisValue[] { UnixMillisecondsNow(), ownerPod, LeaseDurationMs, RetentionMs }).ConfigureAwait(false);
if (result != 1)
{
_logger.LogWarning(
"Pod {OwnerPod} no longer owns transcode session {PlaySessionId}; lease not renewed.",
ownerPod,
playSessionId);
return false;
}
_logger.LogDebug("Renewed lease for transcode session {PlaySessionId}.", playSessionId);
return true;
}
/// <inheritdoc />
public async Task DeleteAsync(string playSessionId, CancellationToken cancellationToken = default)
{
var key = GetKey(playSessionId);
await _db.KeyDeleteAsync(key).ConfigureAwait(false);
_logger.LogDebug("Deleted transcode session {PlaySessionId} from Redis.", playSessionId);
}
/// <inheritdoc />
public async Task<bool> TryTakeoverAsync(string playSessionId, string claimingPod, CancellationToken cancellationToken = default)
{
var result = (long?)await _db.ScriptEvaluateAsync(
TakeoverScript,
keys: new RedisKey[] { GetKey(playSessionId) },
values: new RedisValue[] { UnixMillisecondsNow(), claimingPod, LeaseDurationMs, RetentionMs }).ConfigureAwait(false);
var succeeded = result == 1;
if (succeeded)
{
_logger.LogInformation("Pod {ClaimingPod} successfully took over transcode session {PlaySessionId}.", claimingPod, playSessionId);
}
return succeeded;
}
/// <inheritdoc />
public async Task<IEnumerable<TranscodeSession>> GetActiveSessionsAsync(CancellationToken cancellationToken = default)
{
var sessions = new List<TranscodeSession>();
var servers = _redis.GetServers();
foreach (var server in servers)
{
if (!server.IsConnected)
{
continue;
}
var keys = new List<RedisKey>();
await foreach (var key in server.KeysAsync(database: _db.Database, pattern: KeyPrefix + "*", pageSize: 1000).WithCancellation(cancellationToken).ConfigureAwait(false))
{
keys.Add(key);
}
var tasks = keys.Select(key => _db.StringGetAsync(key)).ToList();
var values = await Task.WhenAll(tasks).ConfigureAwait(false);
foreach (var raw in values)
{
if (!raw.HasValue)
{
continue;
}
TranscodeSession? session;
try
{
session = JsonSerializer.Deserialize<TranscodeSession>(raw.ToString(), _jsonOptions);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to deserialize transcode session from Redis.");
continue;
}
if (session is not null && session.LeaseExpiresUtc > DateTime.UtcNow)
{
sessions.Add(session);
}
}
}
return sessions;
}
private static string GetKey(string playSessionId) => KeyPrefix + playSessionId;
private static long UnixMillisecondsNow() => DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
private sealed class UnixMillisecondsDateTimeConverter : JsonConverter<DateTime>
{
public override DateTime Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
var milliseconds = reader.TryGetInt64(out var value) ? value : (long)reader.GetDouble();
return DateTimeOffset.FromUnixTimeMilliseconds(milliseconds).UtcDateTime;
}
public override void Write(Utf8JsonWriter writer, DateTime value, JsonSerializerOptions options)
{
var utc = value.Kind switch
{
DateTimeKind.Utc => value,
DateTimeKind.Local => value.ToUniversalTime(),
_ => DateTime.SpecifyKind(value, DateTimeKind.Utc)
};
writer.WriteNumberValue(new DateTimeOffset(utc, TimeSpan.Zero).ToUnixTimeMilliseconds());
}
}
}
@@ -0,0 +1,56 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Controller.MediaEncoding;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using StackExchange.Redis;
namespace Emby.Server.Implementations.MediaEncoding;
/// <summary>
/// Pings the configured Redis transcode session store once at startup so an unreachable store is
/// reported there instead of being discovered as a silent loss of cross-pod takeover.
/// </summary>
public sealed class TranscodeStoreConnectivityProbe : IHostedService
{
private readonly IServiceProvider _serviceProvider;
private readonly ILogger<TranscodeStoreConnectivityProbe> _logger;
/// <summary>
/// Initializes a new instance of the <see cref="TranscodeStoreConnectivityProbe"/> class.
/// </summary>
/// <param name="serviceProvider">The service provider used to resolve the Redis connection.</param>
/// <param name="logger">The logger.</param>
public TranscodeStoreConnectivityProbe(IServiceProvider serviceProvider, ILogger<TranscodeStoreConnectivityProbe> logger)
{
_serviceProvider = serviceProvider;
_logger = logger;
}
/// <inheritdoc />
public async Task StartAsync(CancellationToken cancellationToken)
{
try
{
// Resolved here rather than injected: connecting must not be able to abort startup.
var redis = _serviceProvider.GetRequiredService<IConnectionMultiplexer>();
var roundTrip = await redis.GetDatabase().PingAsync().ConfigureAwait(false);
_logger.LogInformation(
"Redis transcode session store is reachable ({RoundTripMs}ms round trip). HA transcode takeover is active.",
(long)roundTrip.TotalMilliseconds);
}
catch (Exception ex)
{
_logger.LogError(
ex,
"Redis transcode session store is configured but UNREACHABLE. HA transcode takeover is not working: sessions stay local to this instance and are lost when it restarts. Check {Key}.",
TranscodeStoreOptions.RedisConnectionStringKey);
}
}
/// <inheritdoc />
public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
@@ -395,29 +395,11 @@ namespace Emby.Server.Implementations.Plugins
var url = new Uri(packageInfo.ImageUrl);
imagePath = Path.Join(path, url.Segments[^1]);
var fileStream = AsyncFile.OpenWrite(imagePath);
Stream? downloadStream = null;
try
// The catalog is refreshed on every dashboard visit and rewrites the manifest of
// every installed plugin, so only fetch an image that is actually missing.
if (!ImageExists(imagePath))
{
downloadStream = await HttpClientFactory
.CreateClient(NamedClient.Default)
.GetStreamAsync(url)
.ConfigureAwait(false);
await downloadStream.CopyToAsync(fileStream).ConfigureAwait(false);
}
catch (HttpRequestException ex)
{
_logger.LogError(ex, "Failed to download image to path {Path} on disk.", imagePath);
imagePath = string.Empty;
}
finally
{
await fileStream.DisposeAsync().ConfigureAwait(false);
if (downloadStream is not null)
{
await downloadStream.DisposeAsync().ConfigureAwait(false);
}
imagePath = await DownloadImage(url, imagePath).ConfigureAwait(false);
}
}
@@ -456,6 +438,67 @@ namespace Emby.Server.Implementations.Plugins
}
}
private static bool ImageExists(string imagePath)
{
var image = new FileInfo(imagePath);
// A previous download may have been interrupted, leaving an empty file behind.
return image.Exists && image.Length > 0;
}
private async Task<string> DownloadImage(Uri url, string imagePath)
{
// Download to a temporary file and move it into place, so that neither a failed download
// nor a concurrent one can be observed as a partially written image.
var tempPath = imagePath + "." + Path.GetRandomFileName();
try
{
var fileStream = AsyncFile.Create(tempPath);
Stream? downloadStream = null;
try
{
downloadStream = await HttpClientFactory
.CreateClient(NamedClient.Default)
.GetStreamAsync(url)
.ConfigureAwait(false);
await downloadStream.CopyToAsync(fileStream).ConfigureAwait(false);
}
finally
{
await fileStream.DisposeAsync().ConfigureAwait(false);
if (downloadStream is not null)
{
await downloadStream.DisposeAsync().ConfigureAwait(false);
}
}
File.Move(tempPath, imagePath, true);
return imagePath;
}
catch (Exception ex) when (ex is HttpRequestException or IOException or UnauthorizedAccessException)
{
_logger.LogError(ex, "Failed to download image to path {Path} on disk.", imagePath);
TryDeleteFile(tempPath);
return string.Empty;
}
}
private void TryDeleteFile(string path)
{
try
{
File.Delete(path);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_logger.LogWarning(ex, "Unable to delete {Path}.", path);
}
}
/// <summary>
/// Reconciles the manifest against any properties that exist locally in a pre-packaged meta.json found at the path.
/// If no file is found, no reconciliation occurs.
@@ -0,0 +1,81 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Controller.ScheduledTasks;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StackExchange.Redis;
namespace Emby.Server.Implementations.ScheduledTasks;
/// <summary>
/// A Redis-backed <see cref="IScanLeaderLease"/> that elects a single scan-leader instance using a
/// TTL lease on a shared key. The lease value is this instance's pod identity; a leader that keeps
/// renewing retains the lease, and any instance can claim it once the previous leader's lease expires.
/// </summary>
public sealed class RedisScanLeaderLease : IScanLeaderLease
{
private const string LeaderKey = "jellyfin:scanleader";
/// <summary>
/// Lua script for atomic acquire-or-renew: if the key is unset (missing or already expired) it is
/// set to this pod for the lease duration and 1 is returned; if it already holds this pod the TTL is
/// extended and 1 is returned; otherwise another pod owns a live lease and 0 is returned.
/// </summary>
private const string AcquireOrRenewScript = @"
local current = redis.call('GET', KEYS[1])
if not current then
redis.call('SET', KEYS[1], ARGV[1], 'PX', ARGV[2])
return 1
elseif current == ARGV[1] then
redis.call('PEXPIRE', KEYS[1], ARGV[2])
return 1
else
return 0
end";
private readonly IDatabase _db;
private readonly ScanLeaderOptions _options;
private readonly ILogger<RedisScanLeaderLease> _logger;
private readonly string _podId;
/// <summary>
/// Initializes a new instance of the <see cref="RedisScanLeaderLease"/> class.
/// </summary>
/// <param name="redis">The Redis connection multiplexer.</param>
/// <param name="options">The scan-leader configuration options.</param>
/// <param name="logger">The logger.</param>
public RedisScanLeaderLease(
IConnectionMultiplexer redis,
IOptions<ScanLeaderOptions> options,
ILogger<RedisScanLeaderLease> logger)
{
_db = redis.GetDatabase();
_options = options.Value;
_logger = logger;
_podId = Environment.GetEnvironmentVariable("JELLYFIN_INSTANCE_ID") ?? Environment.MachineName;
}
/// <inheritdoc />
public async Task<bool> TryAcquireOrRenewAsync(CancellationToken cancellationToken = default)
{
var leaseDurationMs = (long)_options.LeaseDurationSeconds * 1000;
try
{
var result = (long?)await _db.ScriptEvaluateAsync(
AcquireOrRenewScript,
keys: new RedisKey[] { LeaderKey },
values: new RedisValue[] { _podId, leaseDurationMs }).ConfigureAwait(false);
return result == 1;
}
catch (Exception ex)
{
// Fail-safe: if Redis is unreachable, treat this instance as the leader so scheduled scans
// keep running. Every instance scanning is preferable to no instance scanning.
_logger.LogWarning(ex, "Scan-leader lease evaluation failed; treating {PodId} as leader.", _podId);
return true;
}
}
}
@@ -13,6 +13,7 @@ using Jellyfin.Data.Events;
using Jellyfin.Extensions.Json;
using MediaBrowser.Common.Configuration;
using MediaBrowser.Common.Extensions;
using MediaBrowser.Controller.ScheduledTasks;
using MediaBrowser.Model.Tasks;
using Microsoft.Extensions.Logging;
@@ -27,6 +28,8 @@ public class ScheduledTaskWorker : IScheduledTaskWorker
private readonly IApplicationPaths _applicationPaths;
private readonly ILogger _logger;
private readonly ITaskManager _taskManager;
private readonly IScanLeaderLease _scanLeaderLease;
private readonly ScanLeaderOptions _scanLeaderOptions;
private readonly Lock _lastExecutionResultSyncLock = new();
private bool _readFromFile;
private TaskResult _lastExecutionResult;
@@ -41,6 +44,8 @@ public class ScheduledTaskWorker : IScheduledTaskWorker
/// <param name="applicationPaths">The application paths.</param>
/// <param name="taskManager">The task manager.</param>
/// <param name="logger">The logger.</param>
/// <param name="scanLeaderLease">The scan-leader lease used to gate periodic library-mutating tasks, or <c>null</c> to disable gating.</param>
/// <param name="scanLeaderOptions">The scan-leader options, or <c>null</c> to disable gating.</param>
/// <exception cref="ArgumentNullException">
/// scheduledTask
/// or
@@ -52,7 +57,13 @@ public class ScheduledTaskWorker : IScheduledTaskWorker
/// or
/// logger.
/// </exception>
public ScheduledTaskWorker(IScheduledTask scheduledTask, IApplicationPaths applicationPaths, ITaskManager taskManager, ILogger logger)
public ScheduledTaskWorker(
IScheduledTask scheduledTask,
IApplicationPaths applicationPaths,
ITaskManager taskManager,
ILogger logger,
IScanLeaderLease scanLeaderLease = null,
ScanLeaderOptions scanLeaderOptions = null)
{
ArgumentNullException.ThrowIfNull(scheduledTask);
ArgumentNullException.ThrowIfNull(applicationPaths);
@@ -63,6 +74,8 @@ public class ScheduledTaskWorker : IScheduledTaskWorker
_applicationPaths = applicationPaths;
_taskManager = taskManager;
_logger = logger;
_scanLeaderLease = scanLeaderLease;
_scanLeaderOptions = scanLeaderOptions;
InitTriggerEvents();
}
@@ -268,6 +281,20 @@ public class ScheduledTaskWorker : IScheduledTaskWorker
trigger.Stop();
if (_scanLeaderLease is not null
&& _scanLeaderOptions is not null
&& _scanLeaderOptions.Enabled
&& _scanLeaderOptions.GatedTaskKeys is not null
&& _scanLeaderOptions.GatedTaskKeys.Contains(ScheduledTask.Key, StringComparer.Ordinal)
&& !await _scanLeaderLease.TryAcquireOrRenewAsync().ConfigureAwait(false))
{
_logger.LogDebug("Skipping gated task {Task}: this instance does not hold the scan-leader lease.", Name);
// Re-arm the trigger for the next interval without enqueueing on this instance.
trigger.Start(LastExecutionResult, _logger, Name, false);
return;
}
_taskManager.QueueScheduledTask(ScheduledTask, trigger.TaskOptions);
await Task.Delay(1000).ConfigureAwait(false);
@@ -5,8 +5,10 @@ using System.Linq;
using System.Threading.Tasks;
using Jellyfin.Data.Events;
using MediaBrowser.Common.Configuration;
using MediaBrowser.Controller.ScheduledTasks;
using MediaBrowser.Model.Tasks;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace Emby.Server.Implementations.ScheduledTasks;
@@ -23,18 +25,26 @@ public class TaskManager : ITaskManager
private readonly IApplicationPaths _applicationPaths;
private readonly ILogger<TaskManager> _logger;
private readonly IScanLeaderLease? _scanLeaderLease;
private readonly ScanLeaderOptions? _scanLeaderOptions;
/// <summary>
/// Initializes a new instance of the <see cref="TaskManager" /> class.
/// </summary>
/// <param name="applicationPaths">The application paths.</param>
/// <param name="logger">The logger.</param>
/// <param name="scanLeaderLease">The scan-leader lease used to gate periodic library-mutating tasks, or <c>null</c> to disable gating.</param>
/// <param name="scanLeaderOptions">The scan-leader options, or <c>null</c> to disable gating.</param>
public TaskManager(
IApplicationPaths applicationPaths,
ILogger<TaskManager> logger)
ILogger<TaskManager> logger,
IScanLeaderLease? scanLeaderLease = null,
IOptions<ScanLeaderOptions>? scanLeaderOptions = null)
{
_applicationPaths = applicationPaths;
_logger = logger;
_scanLeaderLease = scanLeaderLease;
_scanLeaderOptions = scanLeaderOptions?.Value;
ScheduledTasks = [];
}
@@ -175,7 +185,7 @@ public class TaskManager : ITaskManager
/// <inheritdoc />
public void AddTasks(IEnumerable<IScheduledTask> tasks)
{
var list = tasks.Select(t => new ScheduledTaskWorker(t, _applicationPaths, this, _logger));
var list = tasks.Select(t => new ScheduledTaskWorker(t, _applicationPaths, this, _logger, _scanLeaderLease, _scanLeaderOptions));
ScheduledTasks = ScheduledTasks.Concat(list).ToArray();
}
@@ -5,6 +5,7 @@ using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Common.Configuration;
using MediaBrowser.Controller.IO;
using MediaBrowser.Controller.MediaEncoding;
using MediaBrowser.Model.Globalization;
using MediaBrowser.Model.IO;
using MediaBrowser.Model.Tasks;
@@ -21,6 +22,7 @@ public class DeleteTranscodeFileTask : IScheduledTask, IConfigurableScheduledTas
private readonly IConfigurationManager _configurationManager;
private readonly IFileSystem _fileSystem;
private readonly ILocalizationManager _localization;
private readonly ITranscodeSessionStore _sessionStore;
/// <summary>
/// Initializes a new instance of the <see cref="DeleteTranscodeFileTask"/> class.
@@ -29,16 +31,19 @@ public class DeleteTranscodeFileTask : IScheduledTask, IConfigurableScheduledTas
/// <param name="fileSystem">Instance of the <see cref="IFileSystem"/> interface.</param>
/// <param name="configurationManager">Instance of the <see cref="IConfigurationManager"/> interface.</param>
/// <param name="localization">Instance of the <see cref="ILocalizationManager"/> interface.</param>
/// <param name="sessionStore">Instance of the <see cref="ITranscodeSessionStore"/> interface.</param>
public DeleteTranscodeFileTask(
ILogger<DeleteTranscodeFileTask> logger,
IFileSystem fileSystem,
IConfigurationManager configurationManager,
ILocalizationManager localization)
ILocalizationManager localization,
ITranscodeSessionStore sessionStore)
{
_logger = logger;
_fileSystem = fileSystem;
_configurationManager = configurationManager;
_localization = localization;
_sessionStore = sessionStore;
}
/// <inheritdoc />
@@ -78,25 +83,39 @@ public class DeleteTranscodeFileTask : IScheduledTask, IConfigurableScheduledTas
}
/// <inheritdoc />
public Task ExecuteAsync(IProgress<double> progress, CancellationToken cancellationToken)
public async Task ExecuteAsync(IProgress<double> progress, CancellationToken cancellationToken)
{
var minDateModified = DateTime.UtcNow.AddDays(-1);
progress.Report(50);
DeleteTempFilesFromDirectory(_configurationManager.GetTranscodePath(), minDateModified, progress, cancellationToken);
IEnumerable<TranscodeSession> activeSessions;
try
{
activeSessions = await _sessionStore.GetActiveSessionsAsync(cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to retrieve active transcode sessions. Skipping deletion to avoid removing files in use.");
progress.Report(100);
return;
}
return Task.CompletedTask;
DeleteTempFilesFromDirectory(_configurationManager.GetTranscodePath(), minDateModified, activeSessions, progress, cancellationToken);
}
/// <summary>
/// Deletes the transcoded temp files from directory with a last write time less than a given date.
/// Deletes the transcoded temp files from directory with a last write time less than a given date,
/// skipping any files that belong to an active transcode session.
/// </summary>
/// <param name="directory">The directory.</param>
/// <param name="minDateModified">The min date modified.</param>
/// <param name="activeSessions">The currently active transcode sessions.</param>
/// <param name="progress">The progress.</param>
/// <param name="cancellationToken">The task cancellation token.</param>
private void DeleteTempFilesFromDirectory(string directory, DateTime minDateModified, IProgress<double> progress, CancellationToken cancellationToken)
private void DeleteTempFilesFromDirectory(string directory, DateTime minDateModified, IEnumerable<TranscodeSession> activeSessions, IProgress<double> progress, CancellationToken cancellationToken)
{
var activeSessionList = activeSessions.ToList();
var filesToDelete = _fileSystem.GetFiles(directory, true)
.Where(f => _fileSystem.GetLastWriteTimeUtc(f) < minDateModified)
.ToList();
@@ -112,6 +131,13 @@ public class DeleteTranscodeFileTask : IScheduledTask, IConfigurableScheduledTas
cancellationToken.ThrowIfCancellationRequested();
if (IsFileProtectedByActiveSession(file.FullName, activeSessionList))
{
_logger.LogDebug("Skipping deletion of {FilePath} as it belongs to an active transcode session.", file.FullName);
index++;
continue;
}
FileSystemHelper.DeleteFile(_fileSystem, file.FullName, _logger);
index++;
@@ -121,4 +147,24 @@ public class DeleteTranscodeFileTask : IScheduledTask, IConfigurableScheduledTas
progress.Report(100);
}
private static bool IsFileProtectedByActiveSession(string filePath, IList<TranscodeSession> activeSessions)
{
foreach (var session in activeSessions)
{
if (!string.IsNullOrEmpty(session.ManifestPath) &&
string.Equals(filePath, session.ManifestPath, StringComparison.OrdinalIgnoreCase))
{
return true;
}
if (!string.IsNullOrEmpty(session.SegmentPathPrefix) &&
filePath.StartsWith(session.SegmentPathPrefix, StringComparison.OrdinalIgnoreCase))
{
return true;
}
}
return false;
}
}
@@ -11,7 +11,7 @@ using Microsoft.Extensions.Logging;
namespace Emby.Server.Implementations.ScheduledTasks.Tasks;
/// <summary>
/// Optimizes Jellyfin's database by issuing a VACUUM command.
/// Optimizes Jellyfin's database by issuing VACUUM and ANALYZE commands.
/// </summary>
public class OptimizeDatabaseTask : IScheduledTask, IConfigurableScheduledTask
{
@@ -82,7 +82,7 @@ public class OptimizeDatabaseTask : IScheduledTask, IConfigurableScheduledTask
return;
}
_logger.LogInformation("Optimizing and vacuuming jellyfin.db...");
_logger.LogInformation("Vacuuming and analyzing jellyfin.db...");
try
{
@@ -4,6 +4,7 @@ using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Emby.Server.Implementations.Library.Validators;
using Jellyfin.Data.Enums;
using Jellyfin.Database.Implementations;
using Jellyfin.Database.Implementations.Entities;
@@ -29,6 +30,7 @@ public class PeopleValidationTask : IScheduledTask, IConfigurableScheduledTask
private readonly IDbContextFactory<JellyfinDbContext> _dbContextFactory;
private readonly IFileSystem _fileSystem;
private readonly ILogger<PeopleValidationTask> _logger;
private readonly ILogger<PeopleValidator> _validatorLogger;
private readonly IItemTypeLookup _itemTypeLookup;
/// <summary>
@@ -39,6 +41,7 @@ public class PeopleValidationTask : IScheduledTask, IConfigurableScheduledTask
/// <param name="dbContextFactory">Instance of the <see cref="IDbContextFactory{TContext}"/> interface.</param>
/// <param name="fileSystem">Instance of the <see cref="IFileSystem"/> interface.</param>
/// <param name="logger">Instance of the <see cref="ILogger{PeopleValidationTask}"/> interface.</param>
/// <param name="validatorLogger">Instance of the <see cref="ILogger{PeopleValidator}"/> interface.</param>
/// <param name="itemTypeLookup">Instance of the <see cref="IItemTypeLookup"/> interface.</param>
public PeopleValidationTask(
ILibraryManager libraryManager,
@@ -46,6 +49,7 @@ public class PeopleValidationTask : IScheduledTask, IConfigurableScheduledTask
IDbContextFactory<JellyfinDbContext> dbContextFactory,
IFileSystem fileSystem,
ILogger<PeopleValidationTask> logger,
ILogger<PeopleValidator> validatorLogger,
IItemTypeLookup itemTypeLookup)
{
_libraryManager = libraryManager;
@@ -53,6 +57,7 @@ public class PeopleValidationTask : IScheduledTask, IConfigurableScheduledTask
_dbContextFactory = dbContextFactory;
_fileSystem = fileSystem;
_logger = logger;
_validatorLogger = validatorLogger;
_itemTypeLookup = itemTypeLookup;
}
@@ -165,7 +170,9 @@ public class PeopleValidationTask : IScheduledTask, IConfigurableScheduledTask
// Phase 2: Validate people (33-66%). Runs after orphaned PeopleBaseItemMap entries are
// cleaned up above, so dead people are removed in a single pass instead of requiring a second run.
IProgress<double> validateProgress = new Progress<double>((val) => progress.Report((val / 3) + 33));
await _libraryManager.ValidatePeopleAsync(validateProgress, cancellationToken).ConfigureAwait(false);
await new PeopleValidator(_libraryManager, _validatorLogger)
.Run(validateProgress, cancellationToken)
.ConfigureAwait(false);
// Phase 3: Refresh images for people missing them (66-100%)
IProgress<double> refreshProgress = new Progress<double>((val) => progress.Report((val / 3) + 66));
@@ -107,6 +107,11 @@ public class PluginUpdateTask : IScheduledTask, IConfigurableScheduledTask
{
_logger.LogError(ex, "Error updating {Name}", package.Name);
}
catch (TimeoutException ex)
{
// One slow download must not abort the updates for the remaining plugins.
_logger.LogError(ex, "Error downloading {Name}", package.Name);
}
catch (InvalidDataException ex)
{
_logger.LogError(ex, "Error updating {Name}", package.Name);
@@ -256,7 +256,7 @@ namespace Emby.Server.Implementations.Session
ArgumentException.ThrowIfNullOrEmpty(deviceId);
var activityDate = DateTime.UtcNow;
var session = GetSessionInfo(appName, appVersion, deviceId, deviceName, remoteEndPoint, user);
var session = await GetSessionInfo(appName, appVersion, deviceId, deviceName, remoteEndPoint, user).ConfigureAwait(false);
var lastActivityDate = session.LastActivityDate;
session.LastActivityDate = activityDate;
@@ -491,7 +491,7 @@ namespace Emby.Server.Implementations.Session
/// <param name="remoteEndPoint">The remote end point.</param>
/// <param name="user">The user.</param>
/// <returns>SessionInfo.</returns>
private SessionInfo GetSessionInfo(
private async Task<SessionInfo> GetSessionInfo(
string appName,
string appVersion,
string deviceId,
@@ -504,7 +504,7 @@ namespace Emby.Server.Implementations.Session
ArgumentException.ThrowIfNullOrEmpty(deviceId);
var key = GetSessionKey(appName, deviceId, user?.Id ?? Guid.Empty);
SessionInfo newSession = CreateSessionInfo(key, appName, appVersion, deviceId, deviceName, remoteEndPoint, user);
SessionInfo newSession = await CreateSessionInfo(key, appName, appVersion, deviceId, deviceName, remoteEndPoint, user).ConfigureAwait(false);
SessionInfo sessionInfo = _activeConnections.GetOrAdd(key, newSession);
if (ReferenceEquals(newSession, sessionInfo))
{
@@ -532,7 +532,7 @@ namespace Emby.Server.Implementations.Session
return sessionInfo;
}
private SessionInfo CreateSessionInfo(
private async Task<SessionInfo> CreateSessionInfo(
string key,
string appName,
string appVersion,
@@ -562,7 +562,7 @@ namespace Emby.Server.Implementations.Session
deviceName = "Network Device";
}
var deviceOptions = _deviceManager.GetDeviceOptions(deviceId) ?? new()
var deviceOptions = await _deviceManager.GetDeviceOptions(deviceId).ConfigureAwait(false) ?? new()
{
DeviceId = deviceId
};
@@ -1768,12 +1768,12 @@ namespace Emby.Server.Implementations.Session
// This should be validated above, but if it isn't don't delete all tokens.
ArgumentException.ThrowIfNullOrEmpty(deviceId);
var existing = _deviceManager.GetDevices(
var existing = (await _deviceManager.GetDevices(
new DeviceQuery
{
DeviceId = deviceId,
UserId = user.Id
}).Items;
}).ConfigureAwait(false)).Items;
foreach (var auth in existing)
{
@@ -1801,12 +1801,12 @@ namespace Emby.Server.Implementations.Session
ArgumentException.ThrowIfNullOrEmpty(accessToken);
var existing = _deviceManager.GetDevices(
var existing = (await _deviceManager.GetDevices(
new DeviceQuery
{
Limit = 1,
AccessToken = accessToken
}).Items;
}).ConfigureAwait(false)).Items;
if (existing.Count > 0)
{
@@ -1845,10 +1845,10 @@ namespace Emby.Server.Implementations.Session
{
CheckDisposed();
var existing = _deviceManager.GetDevices(new DeviceQuery
var existing = await _deviceManager.GetDevices(new DeviceQuery
{
UserId = userId
});
}).ConfigureAwait(false);
foreach (var info in existing.Items)
{
@@ -2045,11 +2045,11 @@ namespace Emby.Server.Implementations.Session
/// <inheritdoc />
public async Task<SessionInfo> GetSessionByAuthenticationToken(string token, string deviceId, string remoteEndpoint)
{
var items = _deviceManager.GetDevices(new DeviceQuery
var items = (await _deviceManager.GetDevices(new DeviceQuery
{
AccessToken = token,
Limit = 1
}).Items;
}).ConfigureAwait(false)).Items;
if (items.Count == 0)
{
+16 -2
View File
@@ -90,6 +90,18 @@ namespace Emby.Server.Implementations.SyncPlay
/// <value>The default ping.</value>
public long DefaultPing { get; } = 500;
/// <summary>
/// Gets the maximum ping, in milliseconds, accepted from a session.
/// </summary>
/// <remarks>
/// Pings are reported by clients and are scaled into the delays used to schedule playback,
/// so an unbounded value lets a single session push the whole group's resume point
/// arbitrarily far out, or overflow the arithmetic entirely. Anything above this is not a
/// usable measurement for synchronisation.
/// </remarks>
/// <value>The maximum ping.</value>
public long MaxPing { get; } = 10000;
/// <summary>
/// Gets the maximum time offset error accepted for dates reported by clients, in milliseconds.
/// </summary>
@@ -438,7 +450,7 @@ namespace Emby.Server.Implementations.SyncPlay
{
if (_participants.TryGetValue(session.Id, out GroupMember value))
{
value.Ping = ping;
value.Ping = Math.Clamp(ping, 0, MaxPing);
}
}
@@ -451,7 +463,9 @@ namespace Emby.Server.Implementations.SyncPlay
max = Math.Max(max, session.Ping);
}
return max;
// A group with no participants has no ping to report. Returning long.MinValue would
// overflow the callers that scale this value into ticks, so fall back to the default.
return max == long.MinValue ? DefaultPing : max;
}
/// <inheritdoc />
@@ -181,8 +181,8 @@ namespace Emby.Server.Implementations.SyncPlay
{
if (existingGroup.GroupId.Equals(request.GroupId))
{
// Restore session.
UpdateSessionsCounter(session.UserId, 1);
// Restore session. The session is already in the group and has already
// been counted, so the counter must not be incremented a second time.
group.SessionJoin(session, request, cancellationToken);
return;
}
@@ -332,8 +332,11 @@ namespace Emby.Server.Implementations.SyncPlay
// Group lock required as Group is not thread-safe.
lock (group)
{
// Make sure that session still belongs to this group.
if (_sessionToGroupMap.TryGetValue(session.Id, out var checkGroup) && !checkGroup.GroupId.Equals(group.GroupId))
// Make sure that session still belongs to this group. The lookup can fail
// outright when the session left while this request was waiting on the group
// lock, which is exactly the case this re-check exists to catch.
if (!_sessionToGroupMap.TryGetValue(session.Id, out var checkGroup)
|| !checkGroup.GroupId.Equals(group.GroupId))
{
// Drop request.
return;
@@ -400,7 +403,7 @@ namespace Emby.Server.Implementations.SyncPlay
// Update sessions counter.
var newSessionsCounter = _activeUsers.AddOrUpdate(
userId,
1,
toAdd,
(_, sessionsCounter) => sessionsCounter + toAdd);
// Should never happen.
@@ -11,7 +11,6 @@ using System.Security.Cryptography;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Jellyfin.Data.Events;
using Jellyfin.Extensions;
using Jellyfin.Extensions.Json;
using MediaBrowser.Common.Configuration;
@@ -34,6 +33,9 @@ namespace Emby.Server.Implementations.Updates
public class InstallationManager : IInstallationManager
{
private static readonly SearchValues<char> InvalidPackageNameChars = SearchValues.Create([.. Path.GetInvalidFileNameChars(), '/', '\\']);
// Budget for the whole package download. The response headers are already bounded by the
// HttpClient timeout; this covers reading the package body, which can be large and slow.
private static readonly TimeSpan PackageDownloadTimeout = TimeSpan.FromMinutes(10);
/// <summary>
/// The logger.
@@ -82,8 +84,8 @@ namespace Emby.Server.Implementations.Updates
IServerConfigurationManager config,
IPluginManager pluginManager)
{
_currentInstallations = new List<(InstallationInfo, CancellationTokenSource)>();
_completedInstallationsInternal = new ConcurrentBag<InstallationInfo>();
_currentInstallations = [];
_completedInstallationsInternal = [];
_logger = logger;
_applicationHost = appHost;
@@ -341,8 +343,9 @@ namespace Emby.Server.Implementations.Updates
_applicationHost.NotifyPendingRestart();
}
catch (OperationCanceledException)
catch (OperationCanceledException) when (linkedToken.IsCancellationRequested)
{
// Only an actually cancelled token is a cancellation.
lock (_currentInstallationsLock)
{
_currentInstallations.Remove(tuple);
@@ -356,7 +359,7 @@ namespace Emby.Server.Implementations.Updates
}
catch (Exception ex)
{
_logger.LogError(ex, "Package installation failed");
_logger.LogError(ex, "Package installation failed: {Name} {Version}", package.Name, package.Version);
lock (_currentInstallationsLock)
{
@@ -546,12 +549,36 @@ namespace Emby.Server.Implementations.Updates
throw new InvalidDataException($"Plugin package name '{package.Name}' resolves outside the plugins directory.");
}
using var response = await _httpClientFactory.CreateClient(NamedClient.Default)
.GetAsync(new Uri(package.SourceUrl), cancellationToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
Stream stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false);
await using (stream.ConfigureAwait(false))
// ResponseHeadersRead keeps the body out of the HttpClient timeout, which otherwise covers
// the whole download; the package gets the longer budget below instead.
using var downloadTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
downloadTokenSource.CancelAfter(PackageDownloadTimeout);
var downloadToken = downloadTokenSource.Token;
var buffer = new MemoryStream();
await using (buffer.ConfigureAwait(false))
{
try
{
using var response = await _httpClientFactory.CreateClient(NamedClient.Default)
.GetAsync(new Uri(package.SourceUrl), HttpCompletionOption.ResponseHeadersRead, downloadToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
// The package is read twice, for the checksum and for the extraction, so it has
// to be buffered: the response stream is not seekable.
await response.Content.CopyToAsync(buffer, downloadToken).ConfigureAwait(false);
}
catch (OperationCanceledException ex) when (!cancellationToken.IsCancellationRequested)
{
// Either our budget above or the HttpClient timeout ran out.
throw new TimeoutException(
$"Downloading the package {package.Name} {package.Version} from {package.SourceUrl} timed out.",
ex);
}
buffer.Position = 0;
Stream stream = buffer;
// CA5351: Do Not Use Broken Cryptographic Algorithms
#pragma warning disable CA5351
cancellationToken.ThrowIfCancellationRequested();
+13 -8
View File
@@ -50,10 +50,10 @@ public class DevicesController : BaseJellyfinApiController
/// <returns>An <see cref="OkResult"/> containing the list of devices.</returns>
[HttpGet]
[ProducesResponseType(StatusCodes.Status200OK)]
public ActionResult<QueryResult<DeviceInfoDto>> GetDevices([FromQuery] Guid? userId)
public async Task<ActionResult<QueryResult<DeviceInfoDto>>> GetDevices([FromQuery] Guid? userId)
{
userId = RequestHelpers.GetUserId(User, userId);
return _deviceManager.GetDevicesForUser(userId);
return await _deviceManager.GetDevicesForUser(userId).ConfigureAwait(false);
}
/// <summary>
@@ -66,9 +66,9 @@ public class DevicesController : BaseJellyfinApiController
[HttpGet("Info")]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public ActionResult<DeviceInfoDto> GetDeviceInfo([FromQuery, Required] string id)
public async Task<ActionResult<DeviceInfoDto>> GetDeviceInfo([FromQuery, Required] string id)
{
var deviceInfo = _deviceManager.GetDevice(id);
var deviceInfo = await _deviceManager.GetDevice(id).ConfigureAwait(false);
if (deviceInfo is null)
{
return NotFound();
@@ -87,9 +87,9 @@ public class DevicesController : BaseJellyfinApiController
[HttpGet("Options")]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public ActionResult<DeviceOptionsDto> GetDeviceOptions([FromQuery, Required] string id)
public async Task<ActionResult<DeviceOptionsDto>> GetDeviceOptions([FromQuery, Required] string id)
{
var deviceInfo = _deviceManager.GetDeviceOptions(id);
var deviceInfo = await _deviceManager.GetDeviceOptions(id).ConfigureAwait(false);
if (deviceInfo is null)
{
return NotFound();
@@ -127,7 +127,12 @@ public class DevicesController : BaseJellyfinApiController
[ProducesResponseType(StatusCodes.Status400BadRequest)]
public async Task<ActionResult> DeleteDevice([FromQuery] string[] id)
{
var devices = id.Select(_deviceManager.GetDevice).ToArray();
var devices = new List<DeviceInfoDto?>(id.Length);
foreach (var deviceId in id)
{
devices.Add(await _deviceManager.GetDevice(deviceId).ConfigureAwait(false));
}
if (devices.Any(f => f is null))
{
return BadRequest();
@@ -135,7 +140,7 @@ public class DevicesController : BaseJellyfinApiController
foreach (var device in devices)
{
var sessions = _deviceManager.GetDevices(new DeviceQuery { DeviceId = device!.Id });
var sessions = await _deviceManager.GetDevices(new DeviceQuery { DeviceId = device!.Id }).ConfigureAwait(false);
foreach (var session in sessions.Items)
{
@@ -29,6 +29,7 @@ using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace Jellyfin.Api.Controllers;
@@ -44,6 +45,8 @@ public class DynamicHlsController : BaseJellyfinApiController
private const EncoderPreset DefaultEventEncoderPreset = EncoderPreset.superfast;
private const TranscodingJobType TranscodingJobType = MediaBrowser.Controller.MediaEncoding.TranscodingJobType.Hls;
private static readonly string _podIdentity = Environment.GetEnvironmentVariable("JELLYFIN_INSTANCE_ID") ?? Environment.MachineName;
private readonly Version _minFFmpegFlacInMp4 = new Version(6, 0);
private readonly Version _minFFmpegX265BframeInFmp4 = new Version(7, 0, 1);
private readonly Version _minFFmpegHlsSegmentOptions = new Version(5, 0);
@@ -60,6 +63,8 @@ public class DynamicHlsController : BaseJellyfinApiController
private readonly IDynamicHlsPlaylistGenerator _dynamicHlsPlaylistGenerator;
private readonly DynamicHlsHelper _dynamicHlsHelper;
private readonly EncodingOptions _encodingOptions;
private readonly ITranscodeSessionStore _transcodeSessionStore;
private readonly TranscodeStoreOptions _transcodeStoreOptions;
/// <summary>
/// Initializes a new instance of the <see cref="DynamicHlsController"/> class.
@@ -75,6 +80,8 @@ public class DynamicHlsController : BaseJellyfinApiController
/// <param name="dynamicHlsHelper">Instance of <see cref="DynamicHlsHelper"/>.</param>
/// <param name="encodingHelper">Instance of <see cref="EncodingHelper"/>.</param>
/// <param name="dynamicHlsPlaylistGenerator">Instance of <see cref="IDynamicHlsPlaylistGenerator"/>.</param>
/// <param name="transcodeSessionStore">Instance of the <see cref="ITranscodeSessionStore"/> interface used to register and renew HLS transcoding session leases in the durable store.</param>
/// <param name="transcodeStoreOptions">The <see cref="TranscodeStoreOptions"/> holding the session lease duration.</param>
public DynamicHlsController(
ILibraryManager libraryManager,
IUserManager userManager,
@@ -86,7 +93,9 @@ public class DynamicHlsController : BaseJellyfinApiController
ILogger<DynamicHlsController> logger,
DynamicHlsHelper dynamicHlsHelper,
EncodingHelper encodingHelper,
IDynamicHlsPlaylistGenerator dynamicHlsPlaylistGenerator)
IDynamicHlsPlaylistGenerator dynamicHlsPlaylistGenerator,
ITranscodeSessionStore transcodeSessionStore,
IOptions<TranscodeStoreOptions> transcodeStoreOptions)
{
_libraryManager = libraryManager;
_userManager = userManager;
@@ -99,6 +108,8 @@ public class DynamicHlsController : BaseJellyfinApiController
_dynamicHlsHelper = dynamicHlsHelper;
_encodingHelper = encodingHelper;
_dynamicHlsPlaylistGenerator = dynamicHlsPlaylistGenerator;
_transcodeSessionStore = transcodeSessionStore;
_transcodeStoreOptions = transcodeStoreOptions.Value;
_encodingOptions = serverConfigurationManager.GetEncodingOptions();
}
@@ -306,15 +317,24 @@ public class DynamicHlsController : BaseJellyfinApiController
// If the playlist doesn't already exist, startup ffmpeg
try
{
var isHaMode = await IsHaTakeoverAsync(playSessionId, cancellationToken).ConfigureAwait(false);
job = await _transcodeManager.StartFfMpeg(
state,
playlistPath,
GetCommandLineArguments(playlistPath, state, true, 0),
GetCommandLineArguments(playlistPath, state, true, 0, isHaMode),
Request.HttpContext.User.GetUserId(),
TranscodingJobType,
cancellationTokenSource)
.ConfigureAwait(false);
job.IsLiveOutput = true;
await RegisterTranscodeSessionAsync(
playSessionId ?? string.Empty,
mediaSourceId ?? string.Empty,
playlistPath,
cancellationToken)
.ConfigureAwait(false);
StartLeaseRenewal(playSessionId ?? string.Empty, cancellationToken);
}
catch
{
@@ -1511,14 +1531,23 @@ public class DynamicHlsController : BaseJellyfinApiController
streamingRequest.StartTimeTicks = streamingRequest.CurrentRuntimeTicks;
var isHaMode = await IsHaTakeoverAsync(streamingRequest.PlaySessionId, cancellationToken).ConfigureAwait(false);
state.WaitForPath = segmentPath;
job = await _transcodeManager.StartFfMpeg(
state,
playlistPath,
GetCommandLineArguments(playlistPath, state, false, segmentId),
GetCommandLineArguments(playlistPath, state, false, segmentId, isHaMode),
Request.HttpContext.User.GetUserId(),
TranscodingJobType,
cancellationTokenSource).ConfigureAwait(false);
await RegisterTranscodeSessionAsync(
streamingRequest.PlaySessionId ?? string.Empty,
streamingRequest.MediaSourceId ?? string.Empty,
playlistPath,
cancellationToken)
.ConfigureAwait(false);
StartLeaseRenewal(streamingRequest.PlaySessionId ?? string.Empty, cancellationToken);
}
catch
{
@@ -1543,6 +1572,31 @@ public class DynamicHlsController : BaseJellyfinApiController
}
}
/// <summary>
/// Gets the segment length ffmpeg is told to use. HA mode shortens segments so a takeover pod
/// has to re-encode less; the configured value is user-editable so it is clamped.
/// </summary>
/// <param name="isHaMode">Whether the session is being resumed from the durable store.</param>
/// <param name="requestedSegmentLength">The segment length requested by the streaming pipeline.</param>
/// <param name="encodingOptions">The encoding options holding the recovery segment length.</param>
/// <returns>The segment length in seconds.</returns>
internal static int GetEffectiveSegmentLength(bool isHaMode, int requestedSegmentLength, EncodingOptions encodingOptions)
=> isHaMode
? Math.Clamp(encodingOptions.RecoverySegmentLengthSeconds, 1, 6)
: requestedSegmentLength;
/// <summary>
/// Gets the ffmpeg <c>hls_list_size</c>. HA mode keeps a bounded rolling buffer; 0 (unbounded)
/// otherwise.
/// </summary>
/// <param name="isHaMode">Whether the session is being resumed from the durable store.</param>
/// <param name="encodingOptions">The encoding options holding the recovery buffer count.</param>
/// <returns>The number of segments to keep in the playlist, or 0 for all of them.</returns>
internal static int GetHlsListSize(bool isHaMode, EncodingOptions encodingOptions)
=> isHaMode
? Math.Clamp(encodingOptions.RecoverySegmentBufferCount, 2, 10)
: 0;
internal static async Task WaitForActiveTranscodingRequests(TranscodingJob? job, CancellationToken cancellationToken)
{
while (job?.ActiveRequestCount > 0)
@@ -1554,6 +1608,116 @@ public class DynamicHlsController : BaseJellyfinApiController
private static double[] GetSegmentLengths(StreamState state)
=> GetSegmentLengthsInternal(state.RunTimeTicks ?? 0, state.SegmentLength);
/// <summary>
/// Determines whether the play session is already present in the durable store, which means
/// another pod owned it and this request is resuming it after a failover.
/// </summary>
private async Task<bool> IsHaTakeoverAsync(string? playSessionId, CancellationToken cancellationToken)
{
if (string.IsNullOrEmpty(playSessionId))
{
return false;
}
try
{
return await _transcodeSessionStore.TryGetAsync(playSessionId, cancellationToken).ConfigureAwait(false) is not null;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to check HA mode for session {PlaySessionId}.", playSessionId);
return false;
}
}
/// <summary>
/// Builds the durable record for an HLS output owned by this instance. The manifest and segment
/// paths have to be real, otherwise <c>DeleteTranscodeFileTask</c> cannot tell which files
/// belong to a live session.
/// </summary>
/// <param name="playSessionId">The play session identifier.</param>
/// <param name="mediaSourceId">The media source identifier.</param>
/// <param name="playlistPath">The absolute path of the HLS playlist this instance is writing.</param>
/// <param name="leaseDuration">The initial lease duration.</param>
/// <returns>The session record to store.</returns>
internal static TranscodeSession CreateSessionRecord(string playSessionId, string mediaSourceId, string playlistPath, TimeSpan leaseDuration)
=> TranscodeSession.CreateForPlaylist(playSessionId, mediaSourceId, _podIdentity, playlistPath, leaseDuration);
private async Task RegisterTranscodeSessionAsync(string playSessionId, string mediaSourceId, string playlistPath, CancellationToken cancellationToken)
{
try
{
var session = CreateSessionRecord(
playSessionId,
mediaSourceId,
playlistPath,
TimeSpan.FromSeconds(_transcodeStoreOptions.LeaseDurationSeconds));
await _transcodeSessionStore.SetAsync(session, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to register HLS session {PlaySessionId} in durable store.", playSessionId);
}
}
private void StartLeaseRenewal(string playSessionId, CancellationToken cancellationToken)
{
var renewalInterval = TimeSpan.FromSeconds(Math.Max(1, _transcodeStoreOptions.LeaseDurationSeconds / 3));
_ = Task.Run(
async () =>
{
var ownsLease = true;
try
{
while (!cancellationToken.IsCancellationRequested)
{
try
{
await Task.Delay(renewalInterval, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
break;
}
try
{
if (!await _transcodeSessionStore.RenewLeaseAsync(playSessionId, _podIdentity, cancellationToken).ConfigureAwait(false))
{
// Another pod owns the session now; its record must not be touched.
ownsLease = false;
break;
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to renew lease for HLS session {PlaySessionId}.", playSessionId);
}
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Lease renewal loop for HLS session {PlaySessionId} encountered an unexpected error.", playSessionId);
}
finally
{
if (ownsLease)
{
try
{
await _transcodeSessionStore.DeleteAsync(playSessionId, CancellationToken.None).ConfigureAwait(false);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete HLS session {PlaySessionId} from durable store.", playSessionId);
}
}
}
},
CancellationToken.None);
}
internal static double[] GetSegmentLengthsInternal(long runtimeTicks, int segmentlength)
{
var segmentLengthTicks = TimeSpan.FromSeconds(segmentlength).Ticks;
@@ -1575,7 +1739,7 @@ public class DynamicHlsController : BaseJellyfinApiController
return segments;
}
private string GetCommandLineArguments(string outputPath, StreamState state, bool isEventPlaylist, int startNumber)
private string GetCommandLineArguments(string outputPath, StreamState state, bool isEventPlaylist, int startNumber, bool isHaMode = false)
{
var videoCodec = _encodingHelper.GetVideoEncoder(state, _encodingOptions);
var threads = EncodingHelper.GetNumberOfThreads(state, _encodingOptions, videoCodec);
@@ -1588,10 +1752,13 @@ public class DynamicHlsController : BaseJellyfinApiController
var outputExtension = EncodingHelper.GetSegmentFileExtension(state.Request.SegmentContainer);
var outputTsArg = outputPrefix + "%d" + outputExtension;
var effectiveSegmentLength = GetEffectiveSegmentLength(isHaMode, state.SegmentLength, _encodingOptions);
var hlsListSize = GetHlsListSize(isHaMode, _encodingOptions);
var segmentFormat = string.Empty;
var segmentContainer = outputExtension.TrimStart('.');
var inputModifier = _encodingHelper.GetInputModifier(state, _encodingOptions, segmentContainer);
var hlsArguments = $"-hls_playlist_type {(isEventPlaylist ? "event" : "vod")} -hls_list_size 0";
var hlsArguments = $"-hls_playlist_type {(isEventPlaylist ? "event" : "vod")} -hls_list_size {hlsListSize}";
if (string.Equals(segmentContainer, "ts", StringComparison.OrdinalIgnoreCase))
{
@@ -1644,10 +1811,10 @@ public class DynamicHlsController : BaseJellyfinApiController
_encodingHelper.GetInputArgument(state, _encodingOptions, segmentContainer),
threads,
mapArgs,
GetVideoArguments(state, startNumber, isEventPlaylist, segmentContainer),
GetVideoArguments(state, startNumber, isEventPlaylist, segmentContainer, effectiveSegmentLength),
GetAudioArguments(state),
maxMuxingQueueSize,
state.SegmentLength.ToString(CultureInfo.InvariantCulture),
effectiveSegmentLength.ToString(CultureInfo.InvariantCulture),
segmentFormat,
startNumber.ToString(CultureInfo.InvariantCulture),
baseUrlParam,
@@ -1784,8 +1951,9 @@ public class DynamicHlsController : BaseJellyfinApiController
/// <param name="startNumber">The first number in the hls sequence.</param>
/// <param name="isEventPlaylist">Whether the playlist is EVENT or VOD.</param>
/// <param name="segmentContainer">The segment container.</param>
/// <param name="segmentLength">The effective segment length in seconds (overrides <see cref="StreamState.SegmentLength"/> when HA mode is active).</param>
/// <returns>The command line arguments for video transcoding.</returns>
private string GetVideoArguments(StreamState state, int startNumber, bool isEventPlaylist, string segmentContainer)
private string GetVideoArguments(StreamState state, int startNumber, bool isEventPlaylist, string segmentContainer, int? segmentLength = null)
{
if (state.VideoStream is null)
{
@@ -1860,7 +2028,7 @@ public class DynamicHlsController : BaseJellyfinApiController
args += _encodingHelper.GetVideoQualityParam(state, codec, _encodingOptions, isEventPlaylist ? DefaultEventEncoderPreset : DefaultVodEncoderPreset);
// Set the key frame params for video encoding to match the hls segment time.
args += _encodingHelper.GetHlsVideoKeyFrameArguments(state, codec, state.SegmentLength, isEventPlaylist, startNumber);
args += _encodingHelper.GetHlsVideoKeyFrameArguments(state, codec, segmentLength ?? state.SegmentLength, isEventPlaylist, startNumber);
// Currently b-frames in libx265 breaks the FMP4-HLS playback on iOS, disable it for now.
if (string.Equals(codec, "libx265", StringComparison.OrdinalIgnoreCase)
@@ -16,6 +16,7 @@ using MediaBrowser.Controller.Configuration;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.IO;
using MediaBrowser.Controller.Library;
using MediaBrowser.Controller.Providers;
using MediaBrowser.Model.Configuration;
using MediaBrowser.Model.Entities;
using Microsoft.AspNetCore.Authorization;
@@ -34,6 +35,7 @@ public class LibraryStructureController : BaseJellyfinApiController
private readonly IServerApplicationPaths _appPaths;
private readonly ILibraryManager _libraryManager;
private readonly ILibraryMonitor _libraryMonitor;
private readonly IDirectoryService _directoryService;
/// <summary>
/// Initializes a new instance of the <see cref="LibraryStructureController"/> class.
@@ -41,14 +43,17 @@ public class LibraryStructureController : BaseJellyfinApiController
/// <param name="serverConfigurationManager">Instance of <see cref="IServerConfigurationManager"/> interface.</param>
/// <param name="libraryManager">Instance of <see cref="ILibraryManager"/> interface.</param>
/// <param name="libraryMonitor">Instance of <see cref="ILibraryMonitor"/> interface.</param>
/// <param name="directoryService">Instance of <see cref="IDirectoryService"/> interface.</param>
public LibraryStructureController(
IServerConfigurationManager serverConfigurationManager,
ILibraryManager libraryManager,
ILibraryMonitor libraryMonitor)
ILibraryMonitor libraryMonitor,
IDirectoryService directoryService)
{
_appPaths = serverConfigurationManager.ApplicationPaths;
_libraryManager = libraryManager;
_libraryMonitor = libraryMonitor;
_directoryService = directoryService;
}
/// <summary>
@@ -178,11 +183,11 @@ public class LibraryStructureController : BaseJellyfinApiController
var tempPath = Path.Combine(
rootFolderPath,
Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture));
Directory.Move(currentPath, tempPath);
_directoryService.Move(currentPath, tempPath);
currentPath = tempPath;
}
Directory.Move(currentPath, newPath);
_directoryService.Move(currentPath, newPath);
}
finally
{
+30 -1
View File
@@ -3,7 +3,9 @@ using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Threading.Tasks;
using Jellyfin.Extensions;
using MediaBrowser.Common.Api;
using MediaBrowser.Common.Plugins;
using MediaBrowser.Common.Updates;
using MediaBrowser.Controller.Configuration;
using MediaBrowser.Model.Updates;
@@ -23,16 +25,22 @@ public class PackageController : BaseJellyfinApiController
{
private readonly IInstallationManager _installationManager;
private readonly IServerConfigurationManager _serverConfigurationManager;
private readonly IPluginManager _pluginManager;
/// <summary>
/// Initializes a new instance of the <see cref="PackageController"/> class.
/// </summary>
/// <param name="installationManager">Instance of the <see cref="IInstallationManager"/> interface.</param>
/// <param name="serverConfigurationManager">Instance of the <see cref="IServerConfigurationManager"/> interface.</param>
public PackageController(IInstallationManager installationManager, IServerConfigurationManager serverConfigurationManager)
/// <param name="pluginManager">Instance of the <see cref="IPluginManager"/> interface.</param>
public PackageController(
IInstallationManager installationManager,
IServerConfigurationManager serverConfigurationManager,
IPluginManager pluginManager)
{
_installationManager = installationManager;
_serverConfigurationManager = serverConfigurationManager;
_pluginManager = pluginManager;
}
/// <summary>
@@ -48,6 +56,13 @@ public class PackageController : BaseJellyfinApiController
[FromRoute, Required] string name,
[FromQuery] Guid? assemblyGuid)
{
// Plugins bundled with the server are not published to any repository, so querying
// the configured repositories for them can only ever fail, and does so slowly.
if (IsBundledPlugin(name, assemblyGuid))
{
return NotFound();
}
var packages = await _installationManager.GetAvailablePackages().ConfigureAwait(false);
var result = _installationManager.FilterPackages(
packages,
@@ -96,6 +111,11 @@ public class PackageController : BaseJellyfinApiController
[FromQuery] string? version,
[FromQuery] string? repositoryUrl)
{
if (IsBundledPlugin(name, assemblyGuid))
{
return NotFound();
}
var packages = await _installationManager.GetAvailablePackages().ConfigureAwait(false);
if (!string.IsNullOrEmpty(repositoryUrl))
{
@@ -161,4 +181,13 @@ public class PackageController : BaseJellyfinApiController
_serverConfigurationManager.SaveConfiguration();
return NoContent();
}
private bool IsBundledPlugin(string name, Guid? assemblyGuid)
{
var plugin = assemblyGuid is Guid id && !id.IsEmpty()
? _pluginManager.GetPlugin(id)
: _pluginManager.Plugins.FirstOrDefault(p => p.Name.Equals(name, StringComparison.OrdinalIgnoreCase));
return plugin?.Instance?.CanUninstall == false;
}
}
@@ -557,7 +557,7 @@ public class SubtitleController : BaseJellyfinApiController
if (!string.IsNullOrEmpty(fallbackFontPath))
{
var fontFile = _fileSystem.GetFiles(fallbackFontPath)
.First(i => string.Equals(i.Name, name, StringComparison.OrdinalIgnoreCase));
.FirstOrDefault(i => string.Equals(i.Name, name, StringComparison.OrdinalIgnoreCase));
var fileSize = fontFile?.Length;
if (fontFile is not null && fileSize is not null && fileSize > 0)
+1
View File
@@ -458,6 +458,7 @@ public class DynamicHlsHelper
{
case VideoRangeType.HLG:
case VideoRangeType.DOVIWithHLG:
case VideoRangeType.DOVIInvalid when string.Equals(state.VideoStream.ColorTransfer, "arib-std-b67", StringComparison.OrdinalIgnoreCase):
builder.Append(",VIDEO-RANGE=HLG");
break;
default:
+3 -2
View File
@@ -61,8 +61,9 @@ public enum VideoRangeType
DOVIWithELHDR10Plus,
/// <summary>
/// Dolby Vision with invalid configuration. e.g. Profile 8 compat id 6.
/// When using this range, the server would assume the video is still HDR10 after removing the Dolby Vision metadata.
/// Dolby Vision with invalid configuration, e.g. Profile 8 compat id 6 or inconsistent base-layer color metadata.
/// The base layer is classified as HDR only when its transfer characteristics signal PQ or HLG.
/// Otherwise, it is classified as SDR.
/// </summary>
DOVIInvalid,
@@ -31,8 +31,6 @@ namespace Jellyfin.Server.Implementations.Devices
private readonly IDbContextFactory<JellyfinDbContext> _dbProvider;
private readonly IUserManager _userManager;
private readonly ConcurrentDictionary<string, ClientCapabilities> _capabilitiesMap = new();
private readonly ConcurrentDictionary<int, Device> _devices;
private readonly ConcurrentDictionary<string, DeviceOptions> _deviceOptions;
/// <summary>
/// Initializes a new instance of the <see cref="DeviceManager"/> class.
@@ -43,23 +41,6 @@ namespace Jellyfin.Server.Implementations.Devices
{
_dbProvider = dbProvider;
_userManager = userManager;
_devices = new ConcurrentDictionary<int, Device>();
_deviceOptions = new ConcurrentDictionary<string, DeviceOptions>();
using var dbContext = _dbProvider.CreateDbContext();
foreach (var device in dbContext.Devices
.OrderBy(d => d.Id)
.AsEnumerable())
{
_devices.TryAdd(device.Id, device);
}
foreach (var deviceOption in dbContext.DeviceOptions
.OrderBy(d => d.Id)
.AsEnumerable())
{
_deviceOptions.TryAdd(deviceOption.DeviceId, deviceOption);
}
}
/// <inheritdoc />
@@ -89,8 +70,6 @@ namespace Jellyfin.Server.Implementations.Devices
await dbContext.SaveChangesAsync().ConfigureAwait(false);
}
_deviceOptions[deviceId] = deviceOptions;
DeviceOptionsUpdated?.Invoke(this, new GenericEventArgs<Tuple<string, DeviceOptions>>(new Tuple<string, DeviceOptions>(deviceId, deviceOptions)));
}
@@ -102,21 +81,24 @@ namespace Jellyfin.Server.Implementations.Devices
{
dbContext.Devices.Add(device);
await dbContext.SaveChangesAsync().ConfigureAwait(false);
_devices.TryAdd(device.Id, device);
}
return device;
}
/// <inheritdoc />
public DeviceOptionsDto? GetDeviceOptions(string deviceId)
public async Task<DeviceOptionsDto?> GetDeviceOptions(string deviceId)
{
if (_deviceOptions.TryGetValue(deviceId, out var deviceOptions))
var dbContext = await _dbProvider.CreateDbContextAsync().ConfigureAwait(false);
await using (dbContext.ConfigureAwait(false))
{
return ToDeviceOptionsDto(deviceOptions);
}
var deviceOptions = await dbContext.DeviceOptions
.AsNoTracking()
.FirstOrDefaultAsync(dev => dev.DeviceId == deviceId)
.ConfigureAwait(false);
return null;
return deviceOptions is null ? null : ToDeviceOptionsDto(deviceOptions);
}
}
/// <inheritdoc />
@@ -133,43 +115,79 @@ namespace Jellyfin.Server.Implementations.Devices
}
/// <inheritdoc />
public DeviceInfoDto? GetDevice(string id)
public async Task<DeviceInfoDto?> GetDevice(string id)
{
var device = _devices.Values.Where(d => d.DeviceId == id).OrderByDescending(d => d.DateLastActivity).FirstOrDefault();
_deviceOptions.TryGetValue(id, out var deviceOption);
var dbContext = await _dbProvider.CreateDbContextAsync().ConfigureAwait(false);
await using (dbContext.ConfigureAwait(false))
{
var device = await dbContext.Devices
.AsNoTracking()
.Where(d => d.DeviceId == id)
.OrderByDescending(d => d.DateLastActivity)
.FirstOrDefaultAsync()
.ConfigureAwait(false);
var deviceInfo = device is null ? null : ToDeviceInfo(device, deviceOption);
return deviceInfo is null ? null : ToDeviceInfoDto(deviceInfo);
if (device is null)
{
return null;
}
var deviceOption = await dbContext.DeviceOptions
.AsNoTracking()
.FirstOrDefaultAsync(dev => dev.DeviceId == id)
.ConfigureAwait(false);
return ToDeviceInfoDto(ToDeviceInfo(device, deviceOption));
}
}
/// <inheritdoc />
public QueryResult<Device> GetDevices(DeviceQuery query)
public async Task<QueryResult<Device>> GetDevices(DeviceQuery query)
{
IEnumerable<Device> devices = _devices.Values
.Where(device => !query.UserId.HasValue || device.UserId.Equals(query.UserId.Value))
.Where(device => query.DeviceId is null || device.DeviceId == query.DeviceId)
.Where(device => query.AccessToken is null || device.AccessToken == query.AccessToken)
.OrderBy(d => d.Id)
.ToList();
var count = devices.Count();
if (query.Skip.HasValue)
var dbContext = await _dbProvider.CreateDbContextAsync().ConfigureAwait(false);
await using (dbContext.ConfigureAwait(false))
{
devices = devices.Skip(query.Skip.Value);
}
IQueryable<Device> filtered = dbContext.Devices.AsNoTracking();
if (query.Limit.HasValue && query.Limit.Value > 0)
{
devices = devices.Take(query.Limit.Value);
}
if (query.UserId.HasValue)
{
filtered = filtered.Where(device => device.UserId.Equals(query.UserId.Value));
}
return new QueryResult<Device>(query.Skip, count, devices.ToList());
if (query.DeviceId is not null)
{
filtered = filtered.Where(device => device.DeviceId == query.DeviceId);
}
if (query.AccessToken is not null)
{
filtered = filtered.Where(device => device.AccessToken == query.AccessToken);
}
// Every filter is an exact match on an indexed column and the table holds one row per
// user and device, so paging the materialised set costs less than a second round trip.
var matched = await filtered.OrderBy(d => d.Id).ToListAsync().ConfigureAwait(false);
IEnumerable<Device> devices = matched;
if (query.Skip.HasValue)
{
devices = devices.Skip(query.Skip.Value);
}
if (query.Limit.HasValue && query.Limit.Value > 0)
{
devices = devices.Take(query.Limit.Value);
}
return new QueryResult<Device>(query.Skip, matched.Count, devices.ToList());
}
}
/// <inheritdoc />
public QueryResult<DeviceInfo> GetDeviceInfos(DeviceQuery query)
public async Task<QueryResult<DeviceInfo>> GetDeviceInfos(DeviceQuery query)
{
var devices = GetDevices(query);
var devices = await GetDevices(query).ConfigureAwait(false);
return new QueryResult<DeviceInfo>(
devices.StartIndex,
@@ -178,38 +196,49 @@ namespace Jellyfin.Server.Implementations.Devices
}
/// <inheritdoc />
public QueryResult<DeviceInfoDto> GetDevicesForUser(Guid? userId)
public async Task<QueryResult<DeviceInfoDto>> GetDevicesForUser(Guid? userId)
{
IEnumerable<Device> devices = _devices.Values
.OrderByDescending(d => d.DateLastActivity)
.ThenBy(d => d.DeviceId);
if (!userId.IsNullOrEmpty())
var dbContext = await _dbProvider.CreateDbContextAsync().ConfigureAwait(false);
await using (dbContext.ConfigureAwait(false))
{
var user = _userManager.GetUserById(userId.Value);
if (user is null)
IEnumerable<Device> devices = await dbContext.Devices
.AsNoTracking()
.OrderByDescending(d => d.DateLastActivity)
.ThenBy(d => d.DeviceId)
.ToListAsync()
.ConfigureAwait(false);
if (!userId.IsNullOrEmpty())
{
throw new ResourceNotFoundException();
var user = _userManager.GetUserById(userId.Value);
if (user is null)
{
throw new ResourceNotFoundException();
}
devices = devices.Where(i => CanAccessDevice(user, i.DeviceId));
}
devices = devices.Where(i => CanAccessDevice(user, i.DeviceId));
var options = await dbContext.DeviceOptions
.AsNoTracking()
.ToDictionaryAsync(dev => dev.DeviceId)
.ConfigureAwait(false);
var array = devices.Select(device =>
{
options.TryGetValue(device.DeviceId, out var option);
return ToDeviceInfo(device, option);
})
.Select(ToDeviceInfoDto)
.ToArray();
return new QueryResult<DeviceInfoDto>(array);
}
var array = devices.Select(device =>
{
_deviceOptions.TryGetValue(device.DeviceId, out var option);
return ToDeviceInfo(device, option);
})
.Select(ToDeviceInfoDto)
.ToArray();
return new QueryResult<DeviceInfoDto>(array);
}
/// <inheritdoc />
public async Task DeleteDevice(Device device)
{
_devices.TryRemove(device.Id, out _);
var dbContext = await _dbProvider.CreateDbContextAsync().ConfigureAwait(false);
await using (dbContext.ConfigureAwait(false))
{
@@ -229,8 +258,6 @@ namespace Jellyfin.Server.Implementations.Devices
dbContext.Devices.Update(device);
await dbContext.SaveChangesAsync().ConfigureAwait(false);
}
_devices[device.Id] = device;
}
/// <inheritdoc />
@@ -6,12 +6,14 @@ using System.Reflection;
using Jellyfin.Database.Implementations;
using Jellyfin.Database.Implementations.DbConfiguration;
using Jellyfin.Database.Implementations.Locking;
using Jellyfin.Database.Providers.PostgreSQL;
using Jellyfin.Database.Providers.Sqlite;
using MediaBrowser.Common.Configuration;
using MediaBrowser.Controller.Configuration;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Npgsql;
using JellyfinDbProviderFactory = System.Func<System.IServiceProvider, Jellyfin.Database.Implementations.IJellyfinDatabaseProvider>;
namespace Jellyfin.Server.Implementations.Extensions;
@@ -24,6 +26,13 @@ public static class ServiceCollectionExtensions
private static IEnumerable<Type> DatabaseProviderTypes()
{
yield return typeof(SqliteDatabaseProvider);
yield return typeof(PostgreSqlDatabaseProvider);
}
private static int GetPoolOption(IEnumerable<CustomDatabaseOption>? options, string key, int defaultValue)
{
var value = options?.FirstOrDefault(o => o.Key.Equals(key, StringComparison.OrdinalIgnoreCase))?.Value;
return int.TryParse(value, out var parsed) ? parsed : defaultValue;
}
private static IDictionary<string, JellyfinDbProviderFactory> GetSupportedDbProviders()
@@ -123,6 +132,50 @@ public static class ServiceCollectionExtensions
serviceCollection.AddSingleton<IJellyfinDatabaseProvider>(providerFactory!);
if (efCoreConfiguration.DatabaseType.Equals("Jellyfin-PostgreSQL", StringComparison.OrdinalIgnoreCase))
{
serviceCollection.AddSingleton<NpgsqlDataSource>(static sp =>
{
var config = sp.GetRequiredService<IServerConfigurationManager>().GetConfiguration<DatabaseConfigurationOptions>("database");
var options = config.CustomProviderOptions?.Options;
var connectionString =
Environment.GetEnvironmentVariable("POSTGRES_CONNECTION_STRING")
?? options
?.FirstOrDefault(o => o.Key.Equals("ConnectionString", StringComparison.OrdinalIgnoreCase))
?.Value
?? config.CustomProviderOptions?.ConnectionString
?? throw new InvalidOperationException(
"No PostgreSQL connection string found. Set the POSTGRES_CONNECTION_STRING environment variable, " +
"or provide it via CustomProviderOptions.Options[\"ConnectionString\"] or CustomProviderOptions.ConnectionString.");
// Support postgresql:// / postgres:// URI format (e.g. DATABASE_URL convention).
// NpgsqlDataSourceBuilder requires ADO.NET key=value format; convert if needed.
if (connectionString.StartsWith("postgresql://", StringComparison.OrdinalIgnoreCase)
|| connectionString.StartsWith("postgres://", StringComparison.OrdinalIgnoreCase))
{
var uri = new Uri(connectionString);
var userInfoParts = uri.UserInfo.Split(':', 2);
connectionString = new NpgsqlConnectionStringBuilder
{
Host = uri.Host,
Port = uri.Port > 0 ? uri.Port : 5432,
Database = uri.AbsolutePath.TrimStart('/'),
Username = userInfoParts.Length > 0 ? Uri.UnescapeDataString(userInfoParts[0]) : null,
Password = userInfoParts.Length > 1 ? Uri.UnescapeDataString(userInfoParts[1]) : null,
}.ToString();
}
var dataSourceBuilder = new NpgsqlDataSourceBuilder(connectionString);
dataSourceBuilder.ConnectionStringBuilder.MinPoolSize = GetPoolOption(options, "MinPoolSize", 2);
dataSourceBuilder.ConnectionStringBuilder.MaxPoolSize = GetPoolOption(options, "MaxPoolSize", 20);
dataSourceBuilder.ConnectionStringBuilder.CommandTimeout = GetPoolOption(options, "CommandTimeout", 30);
return dataSourceBuilder.Build();
});
}
switch (efCoreConfiguration.LockingBehavior)
{
case DatabaseLockingBehaviorTypes.NoLock:
@@ -121,20 +121,22 @@ public sealed partial class BaseItemRepository
{
using var context = _dbProvider.CreateDbContext();
var query = context.ItemValuesMap
.AsNoTracking()
.Where(e => itemValueTypes.Any(w => w == e.ItemValue.Type));
var maps = context.ItemValuesMap.AsNoTracking();
if (withItemTypes.Count > 0)
{
query = query.Where(e => withItemTypes.Contains(e.Item.Type));
maps = maps.Where(e => withItemTypes.Contains(e.Item.Type));
}
if (excludeItemTypes.Count > 0)
{
query = query.Where(e => !excludeItemTypes.Contains(e.Item.Type));
maps = maps.Where(e => !excludeItemTypes.Contains(e.Item.Type));
}
return query.Select(e => e.ItemValue)
return context.ItemValues
.AsNoTracking()
.WhereOneOrMany(itemValueTypes, e => e.Type)
.Where(e => maps.Any(m => m.ItemValueId == e.ItemValueId))
.Select(e => new { e.CleanValue, e.Value })
.GroupBy(e => e.CleanValue)
.Select(g => g.Min(v => v.Value)!)
.ToArray();
@@ -217,9 +219,11 @@ public sealed partial class BaseItemRepository
}
else
{
// The representative is the row no other row in its group sorts before, not MIN(Id):
// PostgreSQL has no min(uuid) aggregate, while comparing two uuids is supported everywhere.
representativeIds = masterQuery
.GroupBy(e => e.PresentationUniqueKey)
.Select(g => g.Min(e => e.Id))
.Where(e => !masterQuery.Any(o => o.PresentationUniqueKey == e.PresentationUniqueKey && o.Id.CompareTo(e.Id) < 0))
.Select(e => e.Id)
.ToList();
}
@@ -317,14 +321,7 @@ public sealed partial class BaseItemRepository
.WhereOneOrMany(cleanNames, ivm => ivm.ItemValue.CleanValue);
var seriesTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Series];
var movieTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Movie];
var episodeTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Episode];
var musicAlbumTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.MusicAlbum];
var musicArtistTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.MusicArtist];
var musicVideoTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.MusicVideo];
var programTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.LiveTvProgram];
var audioTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Audio];
var trailerTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Trailer];
// Rewrite query to avoid SelectMany on navigation properties (which requires SQL APPLY, not supported on SQLite)
// Instead, start from ItemValueMaps and join with BaseItems.
@@ -333,9 +330,9 @@ public sealed partial class BaseItemRepository
scopedItems,
ivm => ivm.ItemId,
e => e.Id,
(ivm, e) => new { CleanName = ivm.ItemValue.CleanValue, e.Type, e.SeriesId })
(ivm, e) => new { CleanName = ivm.ItemValue.CleanValue, e.Type, e.SeriesId, e.Id })
.GroupBy(x => new { x.CleanName, x.Type, x.SeriesId })
.Select(g => new { g.Key.CleanName, g.Key.Type, g.Key.SeriesId, Count = g.Count() })
.Select(g => new { g.Key.CleanName, g.Key.Type, g.Key.SeriesId, Count = g.Select(x => x.Id).Distinct().Count() })
.ToList();
// Only studios and genres pass down from a series to its episodes; an artist credit does not.
@@ -357,46 +354,10 @@ public sealed partial class BaseItemRepository
foreach (var group in rawCounts.GroupBy(x => x.CleanName))
{
var counts = new ItemCounts();
foreach (var row in group)
{
if (row.Type == seriesTypeName)
{
counts.SeriesCount += row.Count;
}
else if (row.Type == movieTypeName)
{
counts.MovieCount += row.Count;
}
else if (row.Type == musicAlbumTypeName)
{
counts.AlbumCount += row.Count;
}
else if (row.Type == musicArtistTypeName)
{
counts.ArtistCount += row.Count;
}
else if (row.Type == musicVideoTypeName)
{
counts.MusicVideoCount += row.Count;
}
else if (row.Type == programTypeName)
{
counts.ProgramCount += row.Count;
}
else if (row.Type == audioTypeName)
{
counts.SongCount += row.Count;
}
else if (row.Type == trailerTypeName)
{
counts.TrailerCount += row.Count;
}
}
var counts = ItemCountBuilder.Build(_itemTypeLookup, group.Select(row => (row.Type, row.Count)));
// Episodes are counted separately: the value is usually only written on the series.
counts.EpisodeCount = episodeCounts.GetValueOrDefault(group.Key);
counts.ItemCount = counts.TotalItemCount();
ItemCountBuilder.SetEpisodeCount(counts, episodeCounts.GetValueOrDefault(group.Key));
countsByCleanName[group.Key] = counts;
}
@@ -405,7 +366,9 @@ public sealed partial class BaseItemRepository
{
if (!countsByCleanName.ContainsKey(cleanName))
{
countsByCleanName[cleanName] = new ItemCounts { EpisodeCount = episodeCount, ItemCount = episodeCount };
var counts = new ItemCounts();
ItemCountBuilder.SetEpisodeCount(counts, episodeCount);
countsByCleanName[cleanName] = counts;
}
}
@@ -96,22 +96,36 @@ public sealed partial class BaseItemRepository
// primary version (PrimaryVersionId is null) so detail pages and actions target it instead
// of an arbitrary alternate. Keep the grouped ids as an IQueryable sub-select; materializing
// to a List would inline one bound parameter per id and hit SQLite's variable cap.
// The representative is the row no other row in its group sorts before, not MIN(Id):
// PostgreSQL has no min(uuid) aggregate, while comparing two uuids is supported everywhere.
// The anti-join reads the filtered set twice, so it has to close over a local that the
// reassignment below cannot reach - capturing dbQuery itself makes the tree self-referential.
var candidates = dbQuery;
var enableGroupByPresentationUniqueKey = EnableGroupByPresentationUniqueKey(filter);
if (enableGroupByPresentationUniqueKey && filter.GroupBySeriesPresentationUniqueKey)
{
var groupedIds = dbQuery.GroupBy(e => new { e.PresentationUniqueKey, e.SeriesPresentationUniqueKey })
.Select(g => g.Where(e => e.PrimaryVersionId == null).Min(e => (Guid?)e.Id) ?? g.Min(e => (Guid?)e.Id));
var groupedIds = candidates
.Where(e => !candidates.Any(o => o.PresentationUniqueKey == e.PresentationUniqueKey
&& o.SeriesPresentationUniqueKey == e.SeriesPresentationUniqueKey
&& ((o.PrimaryVersionId == null && e.PrimaryVersionId != null)
|| ((o.PrimaryVersionId == null) == (e.PrimaryVersionId == null) && o.Id.CompareTo(e.Id) < 0))))
.Select(e => e.Id);
dbQuery = context.BaseItems.AsNoTracking().Where(e => groupedIds.Contains(e.Id));
}
else if (enableGroupByPresentationUniqueKey)
{
var groupedIds = dbQuery.GroupBy(e => e.PresentationUniqueKey)
.Select(g => g.Where(e => e.PrimaryVersionId == null).Min(e => (Guid?)e.Id) ?? g.Min(e => (Guid?)e.Id));
var groupedIds = candidates
.Where(e => !candidates.Any(o => o.PresentationUniqueKey == e.PresentationUniqueKey
&& ((o.PrimaryVersionId == null && e.PrimaryVersionId != null)
|| ((o.PrimaryVersionId == null) == (e.PrimaryVersionId == null) && o.Id.CompareTo(e.Id) < 0))))
.Select(e => e.Id);
dbQuery = context.BaseItems.AsNoTracking().Where(e => groupedIds.Contains(e.Id));
}
else if (filter.GroupBySeriesPresentationUniqueKey)
{
var groupedIds = dbQuery.GroupBy(e => e.SeriesPresentationUniqueKey).Select(e => e.Min(x => x.Id));
var groupedIds = candidates
.Where(e => !candidates.Any(o => o.SeriesPresentationUniqueKey == e.SeriesPresentationUniqueKey && o.Id.CompareTo(e.Id) < 0))
.Select(e => e.Id);
dbQuery = context.BaseItems.AsNoTracking().Where(e => groupedIds.Contains(e.Id));
}
else
@@ -626,18 +626,26 @@ public sealed partial class BaseItemRepository
.ToArray();
var tags = context.ItemValuesMap
.Where(ivm => ivm.ItemValue.Type == ItemValueType.Tags)
.Where(ivm => matchingItemIds.Contains(ivm.ItemId))
.Select(ivm => ivm.ItemValue)
.Join(
context.ItemValues,
ivm => ivm.ItemValueId,
iv => iv.ItemValueId,
(ivm, iv) => new { ivm.ItemId, iv.Type, iv.CleanValue, iv.Value })
.Where(iv => iv.Type == ItemValueType.Tags)
.Where(iv => matchingItemIds.Contains(iv.ItemId))
.GroupBy(iv => iv.CleanValue)
.Select(g => g.Min(iv => iv.Value))
.OrderBy(t => t)
.ToArray();
var genres = context.ItemValuesMap
.Where(ivm => ivm.ItemValue.Type == ItemValueType.Genre)
.Where(ivm => matchingItemIds.Contains(ivm.ItemId))
.Select(ivm => ivm.ItemValue)
.Join(
context.ItemValues,
ivm => ivm.ItemValueId,
iv => iv.ItemValueId,
(ivm, iv) => new { ivm.ItemId, iv.Type, iv.CleanValue, iv.Value })
.Where(iv => iv.Type == ItemValueType.Genre)
.Where(iv => matchingItemIds.Contains(iv.ItemId))
.GroupBy(iv => iv.CleanValue)
.Select(g => g.Min(iv => iv.Value))
.OrderBy(g => g)
@@ -38,22 +38,32 @@ public sealed partial class BaseItemRepository
// Shared by the isPlayed filter and the IsPlayed/IsUnplayed ordering so the two cannot disagree.
private Expression<Func<BaseItemEntity, bool>> BuildIsPlayedFilter(JellyfinDbContext context, User user)
{
var userId = user.Id;
// Folders (Series, Seasons, BoxSets, albums, ...) carry no played state of their own and count
// as played once no descendant is left unplayed.
var unplayedLeafItems = GetAccessFilteredLeafItemsQuery(context, user)
.Where(BuildLeafIsPlayedFilter(context, user.Id).Not());
// Leaf items carry their own played state.
return IsFolderFilter.And(BuildHasDescendantFilter(context, unplayedLeafItems).Not())
.Or(IsFolderFilter.Not().And(BuildLeafIsPlayedFilter(context, user.Id)));
}
private static Expression<Func<BaseItemEntity, bool>> BuildLeafIsPlayedFilter(JellyfinDbContext context, Guid userId)
{
var playedItemIds = context.UserData
.Where(ud => ud.UserId == userId && ud.Played)
.Select(ud => ud.ItemId);
// Folders (Series, Seasons, BoxSets, albums, ...) have none and count as played once no
// descendant is left unplayed, matching what the DTO reports for them. This has to key off
// the item itself rather than off the requested item types: tag and collection listings mix
// folders and leaf items in a single query.
var unplayedLeafItems = GetAccessFilteredLeafItemsQuery(context, user)
.Where(e => !e.UserData!.Any(ud => ud.UserId == userId && ud.Played));
// The primaries of every version group holding a played row, whichever version carries it.
var playedGroupIds = context.BaseItems
.Where(v => v.PrimaryVersionId != null
&& context.UserData.Any(ud => ud.UserId == userId
&& ud.Played
&& (ud.ItemId == v.Id || ud.ItemId == v.PrimaryVersionId)))
.Select(v => v.PrimaryVersionId!.Value);
return IsFolderFilter.And(BuildHasDescendantFilter(context, unplayedLeafItems).Not())
.Or(IsFolderFilter.Not().And(e => playedItemIds.Contains(e.Id)));
return e => playedItemIds.Contains(e.Id)
|| playedGroupIds.Contains(e.Id)
|| (e.PrimaryVersionId != null && playedGroupIds.Contains(e.PrimaryVersionId.Value));
}
// "und" is the language filters' stand-in for a track that declares no language at all.
@@ -0,0 +1,96 @@
using System;
using System.Collections.Generic;
using Jellyfin.Data.Enums;
using MediaBrowser.Controller.Persistence;
using MediaBrowser.Model.Dto;
namespace Jellyfin.Server.Implementations.Item;
/// <summary>
/// Turns per-type counts into an <see cref="ItemCounts"/>.
/// </summary>
internal static class ItemCountBuilder
{
/// <summary>
/// Builds the counts of one by-name item.
/// </summary>
/// <param name="itemTypeLookup">The item type lookup.</param>
/// <param name="counts">The counted items, by type name. A type may repeat.</param>
/// <returns>The counts.</returns>
public static ItemCounts Build(IItemTypeLookup itemTypeLookup, IEnumerable<(string Type, int Count)> counts)
{
ArgumentNullException.ThrowIfNull(itemTypeLookup);
ArgumentNullException.ThrowIfNull(counts);
var lookup = itemTypeLookup.BaseItemKindNames;
var result = new ItemCounts();
foreach (var (type, count) in counts)
{
// Accumulated rather than assigned: a caller may group by something finer than the
// type and hand the same type over more than once.
if (string.Equals(type, lookup[BaseItemKind.MusicAlbum], StringComparison.Ordinal))
{
result.AlbumCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.MusicArtist], StringComparison.Ordinal))
{
result.ArtistCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.Episode], StringComparison.Ordinal))
{
result.EpisodeCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.Movie], StringComparison.Ordinal))
{
result.MovieCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.MusicVideo], StringComparison.Ordinal))
{
result.MusicVideoCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.LiveTvProgram], StringComparison.Ordinal))
{
result.ProgramCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.Series], StringComparison.Ordinal))
{
result.SeriesCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.Audio], StringComparison.Ordinal))
{
result.SongCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.Trailer], StringComparison.Ordinal))
{
result.TrailerCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.BoxSet], StringComparison.Ordinal))
{
result.BoxSetCount += count;
}
else if (string.Equals(type, lookup[BaseItemKind.Book], StringComparison.Ordinal))
{
result.BookCount += count;
}
}
result.ItemCount = result.TotalItemCount();
return result;
}
/// <summary>
/// Replaces the episode count, which both by-name paths decide separately from the other
/// types because a genre or studio is usually written on the series rather than its episodes.
/// </summary>
/// <param name="counts">The counts to update.</param>
/// <param name="episodeCount">The episode count.</param>
public static void SetEpisodeCount(ItemCounts counts, int episodeCount)
{
ArgumentNullException.ThrowIfNull(counts);
counts.EpisodeCount = episodeCount;
counts.ItemCount = counts.TotalItemCount();
}
}
@@ -7,6 +7,7 @@ using System.Linq;
using Jellyfin.Data.Enums;
using Jellyfin.Database.Implementations;
using Jellyfin.Database.Implementations.Entities;
using Jellyfin.Database.Implementations.Enums;
using Jellyfin.Extensions;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Persistence;
@@ -125,178 +126,286 @@ public class ItemCountService : IItemCountService
/// <inheritdoc />
public ItemCounts GetItemCountsForNameItem(BaseItemKind kind, Guid id, BaseItemKind[] relatedItemKinds, InternalItemsQuery accessFilter)
{
using var context = _dbProvider.CreateDbContext();
return GetItemCountsForNameItems(kind, [id], relatedItemKinds, accessFilter)[id];
}
var item = context.BaseItems.AsNoTracking()
.Where(e => e.Id == id)
.Select(e => new { e.Name, e.CleanName })
.FirstOrDefault();
if (item is null)
private static ItemValueType[] GetItemValueTypes(BaseItemKind kind)
=> kind switch
{
return new ItemCounts();
BaseItemKind.MusicArtist => [ItemValueType.Artist, ItemValueType.AlbumArtist],
BaseItemKind.Genre or BaseItemKind.MusicGenre => [ItemValueType.Genre],
BaseItemKind.Studio => [ItemValueType.Studios],
_ => []
};
/// <inheritdoc />
public Dictionary<Guid, ItemCounts> GetItemCountsForNameItems(BaseItemKind kind, IReadOnlyList<Guid> ids, BaseItemKind[] relatedItemKinds, InternalItemsQuery accessFilter)
{
ArgumentNullException.ThrowIfNull(ids);
ArgumentNullException.ThrowIfNull(relatedItemKinds);
ArgumentNullException.ThrowIfNull(accessFilter);
var result = new Dictionary<Guid, ItemCounts>();
if (ids.Count == 0)
{
return result;
}
IQueryable<BaseItemEntity> baseQuery;
switch (kind)
{
case BaseItemKind.Person:
baseQuery = ItemsById(context, context.PeopleBaseItemMap
.AsNoTracking()
.Where(m => m.People.Name == item.Name)
.Select(m => m.ItemId));
break;
case BaseItemKind.MusicArtist:
baseQuery = ItemsById(context, context.ItemValuesMap
.AsNoTracking()
.Where(ivm => ivm.ItemValue.CleanValue == item.CleanName
&& (ivm.ItemValue.Type == ItemValueType.Artist || ivm.ItemValue.Type == ItemValueType.AlbumArtist))
.Select(ivm => ivm.ItemId));
break;
case BaseItemKind.Genre:
case BaseItemKind.MusicGenre:
baseQuery = ItemsById(context, context.ItemValuesMap
.AsNoTracking()
.Where(ivm => ivm.ItemValue.CleanValue == item.CleanName
&& ivm.ItemValue.Type == ItemValueType.Genre)
.Select(ivm => ivm.ItemId));
break;
case BaseItemKind.Studio:
baseQuery = ItemsById(context, context.ItemValuesMap
.AsNoTracking()
.Where(ivm => ivm.ItemValue.CleanValue == item.CleanName
&& ivm.ItemValue.Type == ItemValueType.Studios)
.Select(ivm => ivm.ItemId));
break;
case BaseItemKind.Year:
if (int.TryParse(item.Name, NumberStyles.Integer, CultureInfo.InvariantCulture, out var year))
{
baseQuery = context.BaseItems
.AsNoTracking()
.Where(e => e.ProductionYear == year);
}
else
{
return new ItemCounts();
}
using var context = _dbProvider.CreateDbContext();
break;
default:
return new ItemCounts();
var idsArray = ids as Guid[] ?? ids.ToArray();
var nameItems = context.BaseItems.AsNoTracking()
.WhereOneOrMany(idsArray, e => e.Id)
.Select(e => new NameItem(e.Id, e.Name, e.CleanName))
.ToArray();
foreach (var id in ids)
{
result[id] = new ItemCounts();
}
if (nameItems.Length == 0)
{
return result;
}
var typeNames = relatedItemKinds.Select(k => _itemTypeLookup.BaseItemKindNames[k]).ToArray();
baseQuery = baseQuery.Where(e => typeNames.Contains(e.Type));
var related = _queryHelpers.ApplyAccessFiltering(
context,
context.BaseItems.AsNoTracking().Where(e => typeNames.Contains(e.Type)),
accessFilter);
baseQuery = _queryHelpers.ApplyAccessFiltering(context, baseQuery, accessFilter);
var counts = baseQuery
.GroupBy(x => x.Type)
.Select(x => new { x.Key, Count = x.Count() })
.ToArray();
var lookup = _itemTypeLookup.BaseItemKindNames;
var result = new ItemCounts();
var totalCount = 0;
foreach (var count in counts)
var valueTypes = GetItemValueTypes(kind);
if (valueTypes.Length > 0)
{
totalCount += count.Count;
if (string.Equals(count.Key, lookup[BaseItemKind.MusicAlbum], StringComparison.Ordinal))
{
result.AlbumCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.MusicArtist], StringComparison.Ordinal))
{
result.ArtistCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.Episode], StringComparison.Ordinal))
{
result.EpisodeCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.Movie], StringComparison.Ordinal))
{
result.MovieCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.MusicVideo], StringComparison.Ordinal))
{
result.MusicVideoCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.LiveTvProgram], StringComparison.Ordinal))
{
result.ProgramCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.Series], StringComparison.Ordinal))
{
result.SeriesCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.Audio], StringComparison.Ordinal))
{
result.SongCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.Trailer], StringComparison.Ordinal))
{
result.TrailerCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.BoxSet], StringComparison.Ordinal))
{
result.BoxSetCount = count.Count;
}
else if (string.Equals(count.Key, lookup[BaseItemKind.Book], StringComparison.Ordinal))
{
result.BookCount = count.Count;
}
CountByItemValue(context, related, kind, relatedItemKinds, valueTypes, nameItems, result);
}
if (kind is BaseItemKind.Studio or BaseItemKind.Genre or BaseItemKind.MusicGenre
&& relatedItemKinds.Contains(BaseItemKind.Episode)
&& relatedItemKinds.Contains(BaseItemKind.Series))
else if (kind == BaseItemKind.Person)
{
var rolledUpEpisodeCount = CountEpisodesOfTaggedSeries(context, baseQuery, accessFilter, out var directEpisodeCount);
totalCount += rolledUpEpisodeCount - result.EpisodeCount + directEpisodeCount;
result.EpisodeCount = rolledUpEpisodeCount + directEpisodeCount;
CountByPersonName(context, related, nameItems, result);
}
else if (kind == BaseItemKind.Year)
{
CountByProductionYear(related, nameItems, result);
}
result.ItemCount = totalCount;
return result;
}
private int CountEpisodesOfTaggedSeries(
private void CountByItemValue(
JellyfinDbContext context,
IQueryable<BaseItemEntity> taggedItems,
InternalItemsQuery accessFilter,
out int unrelatedEpisodeCount)
IQueryable<BaseItemEntity> related,
BaseItemKind kind,
BaseItemKind[] relatedItemKinds,
ItemValueType[] valueTypes,
NameItem[] nameItems,
Dictionary<Guid, ItemCounts> result)
{
var cleanNames = nameItems
.Select(n => n.CleanName)
.OfType<string>()
.Distinct(StringComparer.Ordinal)
.ToArray();
if (cleanNames.Length == 0)
{
return;
}
var grouped = context.ItemValuesMap.AsNoTracking()
.Where(ivm => valueTypes.Contains(ivm.ItemValue.Type))
.WhereOneOrMany(cleanNames, ivm => ivm.ItemValue.CleanValue)
.Join(related, ivm => ivm.ItemId, e => e.Id, (ivm, e) => new { ivm.ItemValue.CleanValue, e.Type, e.Id })
.GroupBy(x => new { x.CleanValue, x.Type })
.Select(g => new { g.Key.CleanValue, g.Key.Type, Count = g.Select(x => x.Id).Distinct().Count() })
.ToArray();
var byCleanName = grouped
.GroupBy(g => g.CleanValue, StringComparer.Ordinal)
.ToDictionary(g => g.Key, g => g.Select(x => (x.Type, x.Count)).ToArray(), StringComparer.Ordinal);
var seriesTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Series];
var episodeRollUp = RollsUpEpisodes(kind, relatedItemKinds)
&& Array.Exists(grouped, g => string.Equals(g.Type, seriesTypeName, StringComparison.Ordinal))
? CountEpisodesOfTaggedSeriesByCleanName(context, related, valueTypes, cleanNames)
: null;
foreach (var nameItem in nameItems)
{
if (nameItem.CleanName is null || !byCleanName.TryGetValue(nameItem.CleanName, out var counts))
{
continue;
}
var itemCounts = ItemCountBuilder.Build(_itemTypeLookup, counts);
if (episodeRollUp is not null)
{
var rollUp = episodeRollUp.GetValueOrDefault(nameItem.CleanName);
// Episodes of a tagged series count towards it even when untagged themselves, and
// a tagged episode of a tagged series must not be counted a second time.
var directEpisodeCount = itemCounts.EpisodeCount - rollUp.TaggedEpisodesOfTaggedSeries;
ItemCountBuilder.SetEpisodeCount(itemCounts, rollUp.EpisodesOfTaggedSeries + directEpisodeCount);
}
result[nameItem.Id] = itemCounts;
}
}
private void CountByPersonName(
JellyfinDbContext context,
IQueryable<BaseItemEntity> related,
NameItem[] nameItems,
Dictionary<Guid, ItemCounts> result)
{
var names = nameItems
.Select(n => n.Name)
.OfType<string>()
.Distinct(StringComparer.Ordinal)
.ToArray();
if (names.Length == 0)
{
return;
}
var grouped = context.PeopleBaseItemMap.AsNoTracking()
.WhereOneOrMany(names, m => m.People.Name)
.Join(related, m => m.ItemId, e => e.Id, (m, e) => new { m.People.Name, e.Type, e.Id })
.GroupBy(x => new { x.Name, x.Type })
// A person can be credited on one item more than once, in different roles.
.Select(g => new { g.Key.Name, g.Key.Type, Count = g.Select(x => x.Id).Distinct().Count() })
.ToArray();
ApplyGroupedCounts(nameItems, n => n.Name, grouped.Select(g => (g.Name, g.Type, g.Count)), result);
}
private void CountByProductionYear(
IQueryable<BaseItemEntity> related,
NameItem[] nameItems,
Dictionary<Guid, ItemCounts> result)
{
var years = new List<int>();
foreach (var nameItem in nameItems)
{
if (int.TryParse(nameItem.Name, NumberStyles.Integer, CultureInfo.InvariantCulture, out var year)
&& !years.Contains(year))
{
years.Add(year);
}
}
if (years.Count == 0)
{
return;
}
// No join, so no row can be reached twice and a plain count is the distinct count.
var grouped = related
.Where(e => e.ProductionYear != null)
.WhereOneOrMany(years, e => e.ProductionYear!.Value)
.GroupBy(e => new { Year = e.ProductionYear!.Value, e.Type })
.Select(g => new { g.Key.Year, g.Key.Type, Count = g.Count() })
.ToArray();
var byYear = grouped
.GroupBy(g => g.Year)
.ToDictionary(g => g.Key, g => g.Select(x => (x.Type, x.Count)).ToArray());
foreach (var nameItem in nameItems)
{
if (int.TryParse(nameItem.Name, NumberStyles.Integer, CultureInfo.InvariantCulture, out var year)
&& byYear.TryGetValue(year, out var counts))
{
result[nameItem.Id] = ItemCountBuilder.Build(_itemTypeLookup, counts);
}
}
}
private void ApplyGroupedCounts(
NameItem[] nameItems,
Func<NameItem, string?> keySelector,
IEnumerable<(string Key, string Type, int Count)> grouped,
Dictionary<Guid, ItemCounts> result)
{
var byKey = grouped
.GroupBy(g => g.Key, StringComparer.Ordinal)
.ToDictionary(g => g.Key, g => g.Select(x => (x.Type, x.Count)).ToArray(), StringComparer.Ordinal);
foreach (var nameItem in nameItems)
{
var key = keySelector(nameItem);
if (key is not null && byKey.TryGetValue(key, out var counts))
{
result[nameItem.Id] = ItemCountBuilder.Build(_itemTypeLookup, counts);
}
}
}
private static bool RollsUpEpisodes(BaseItemKind kind, BaseItemKind[] relatedItemKinds)
=> kind is BaseItemKind.Studio or BaseItemKind.Genre or BaseItemKind.MusicGenre
&& relatedItemKinds.Contains(BaseItemKind.Episode)
&& relatedItemKinds.Contains(BaseItemKind.Series);
private Dictionary<string, (int EpisodesOfTaggedSeries, int TaggedEpisodesOfTaggedSeries)> CountEpisodesOfTaggedSeriesByCleanName(
JellyfinDbContext context,
IQueryable<BaseItemEntity> related,
ItemValueType[] valueTypes,
string[] cleanNames)
{
var seriesTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Series];
var episodeTypeName = _itemTypeLookup.BaseItemKindNames[BaseItemKind.Episode];
var taggedSeriesIds = taggedItems.Where(e => e.Type == seriesTypeName).Select(e => e.Id);
unrelatedEpisodeCount = taggedItems.Count(e => e.Type == episodeTypeName
&& (e.SeriesId == null || !taggedSeriesIds.Contains(e.SeriesId.Value)));
var taggedValues = context.ItemValuesMap.AsNoTracking()
.Where(ivm => valueTypes.Contains(ivm.ItemValue.Type))
.WhereOneOrMany(cleanNames, ivm => ivm.ItemValue.CleanValue);
// Materialised so the episode count drives off IX_BaseItems_SeriesId.
var seriesIds = taggedItems
.Where(e => e.Type == seriesTypeName)
.Select(e => e.Id)
// The series carrying each clean name. Distinct, because one item can be mapped to the
// same clean name once per value type.
var taggedSeries = taggedValues
.Join(
related.Where(e => e.Type == seriesTypeName),
ivm => ivm.ItemId,
e => e.Id,
(ivm, e) => new { ivm.ItemValue.CleanValue, SeriesId = e.Id })
.Distinct();
var episodes = related.Where(e => e.Type == episodeTypeName && e.SeriesId != null);
var episodesOfTaggedSeries = taggedSeries
.Join(episodes, s => s.SeriesId, e => e.SeriesId!.Value, (s, e) => new { s.CleanValue, e.Id })
.GroupBy(x => x.CleanValue)
.Select(g => new { CleanValue = g.Key, Count = g.Select(x => x.Id).Distinct().Count() })
.ToArray();
if (seriesIds.Length == 0)
// Episodes that carry the clean name themselves *and* belong to a series carrying it. The
// roll-up already counts those, so they have to come off the directly tagged ones.
var taggedEpisodesOfTaggedSeries = taggedValues
.Join(episodes, ivm => ivm.ItemId, e => e.Id, (ivm, e) => new { ivm.ItemValue.CleanValue, e.Id, e.SeriesId })
.Join(
taggedSeries,
e => new { e.CleanValue, SeriesId = e.SeriesId!.Value },
s => new { s.CleanValue, s.SeriesId },
(e, s) => new { e.CleanValue, e.Id })
.GroupBy(x => x.CleanValue)
.Select(g => new { CleanValue = g.Key, Count = g.Select(x => x.Id).Distinct().Count() })
.ToArray();
var taggedLookup = taggedEpisodesOfTaggedSeries
.ToDictionary(x => x.CleanValue, x => x.Count, StringComparer.Ordinal);
// Every clean name in taggedLookup came from an episode of a tagged series, so it always
// has a row in episodesOfTaggedSeries too - no second merge pass is needed.
var result = new Dictionary<string, (int EpisodesOfTaggedSeries, int TaggedEpisodesOfTaggedSeries)>(StringComparer.Ordinal);
foreach (var entry in episodesOfTaggedSeries)
{
return 0;
result[entry.CleanValue] = (entry.Count, taggedLookup.GetValueOrDefault(entry.CleanValue));
}
var episodes = context.BaseItems.AsNoTracking()
.Where(e => e.Type == episodeTypeName && e.SeriesId != null)
.WhereOneOrMany(seriesIds, e => e.SeriesId!.Value);
return _queryHelpers.ApplyAccessFiltering(context, episodes, accessFilter).Count();
return result;
}
private static IQueryable<BaseItemEntity> ItemsById(JellyfinDbContext context, IQueryable<Guid> itemIds)
=> context.BaseItems.AsNoTracking().Where(e => itemIds.Contains(e.Id));
/// <inheritdoc/>
public int GetPlayedCount(InternalItemsQuery filter, Guid ancestorId)
{
@@ -622,4 +731,12 @@ public class ItemCountService : IItemCountService
return result is null ? (0, 0) : (result.Played, result.Total);
}
/// <summary>
/// A by-name item, reduced to the three columns the counting keys off.
/// </summary>
/// <param name="Id">The id of the by-name item.</param>
/// <param name="Name">The name of the by-name item.</param>
/// <param name="CleanName">The cleaned name of the by-name item.</param>
private sealed record NameItem(Guid Id, string? Name, string? CleanName);
}
@@ -176,14 +176,6 @@ public class ItemPersistenceService : IItemPersistenceService
var context = await _dbProvider.CreateDbContextAsync(cancellationToken).ConfigureAwait(false);
await using (context.ConfigureAwait(false))
{
if (!await context.BaseItems
.AnyAsync(bi => bi.Id == item.Id, cancellationToken)
.ConfigureAwait(false))
{
_logger.LogWarning("Unable to save ImageInfo for non existing BaseItem");
return;
}
await context.BaseItemImageInfos
.Where(e => e.ItemId == item.Id)
.ExecuteDeleteAsync(cancellationToken)
@@ -193,7 +185,26 @@ public class ItemPersistenceService : IItemPersistenceService
.AddRangeAsync(images, cancellationToken)
.ConfigureAwait(false);
await context.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
try
{
await context.SaveChangesAsync(cancellationToken).ConfigureAwait(false);
}
catch (DbUpdateException)
{
// Checking that the item exists before writing leaves a gap a scan can delete it
// through, turning the insert into a foreign key violation that fails the whole
// refresh instead of the no-op intended here. Let the insert be the check: it is the
// only point at which the answer cannot go stale. Nothing is orphaned by the delete
// above, because deleting the item cascades to its images anyway.
if (await context.BaseItems
.AnyAsync(bi => bi.Id == item.Id, cancellationToken)
.ConfigureAwait(false))
{
throw;
}
_logger.LogWarning("Unable to save ImageInfo for non existing BaseItem {ItemId}", item.Id);
}
}
}
@@ -35,6 +35,7 @@
<ProjectReference Include="..\MediaBrowser.Controller\MediaBrowser.Controller.csproj" />
<ProjectReference Include="..\MediaBrowser.Model\MediaBrowser.Model.csproj" />
<ProjectReference Include="..\src\Jellyfin.Database\Jellyfin.Database.Implementations\Jellyfin.Database.Implementations.csproj" />
<ProjectReference Include="..\src\Jellyfin.Database\Jellyfin.Database.Providers.PostgreSQL\Jellyfin.Database.Providers.PostgreSQL.csproj" />
<ProjectReference Include="..\src\Jellyfin.Database\Jellyfin.Database.Providers.Sqlite\Jellyfin.Database.Providers.Sqlite.csproj" />
</ItemGroup>
@@ -126,95 +126,95 @@ namespace Jellyfin.Server.Implementations.Security
return authInfo;
}
var device = (await _deviceManager.GetDevices(
new DeviceQuery { AccessToken = token }).ConfigureAwait(false)).Items.FirstOrDefault();
if (device is not null)
{
authInfo.IsAuthenticated = true;
var updateToken = false;
// TODO: Remove these checks for IsNullOrWhiteSpace
if (string.IsNullOrWhiteSpace(authInfo.Client))
{
authInfo.Client = device.AppName;
}
if (string.IsNullOrWhiteSpace(authInfo.DeviceId))
{
authInfo.DeviceId = device.DeviceId;
}
// Temporary. TODO - allow clients to specify that the token has been shared with a casting device
var allowTokenInfoUpdate = !authInfo.Client.Contains("chromecast", StringComparison.OrdinalIgnoreCase);
if (string.IsNullOrWhiteSpace(authInfo.Device))
{
authInfo.Device = device.DeviceName;
}
else if (!string.Equals(authInfo.Device, device.DeviceName, StringComparison.OrdinalIgnoreCase))
{
if (allowTokenInfoUpdate)
{
updateToken = true;
device.DeviceName = authInfo.Device;
}
}
if (string.IsNullOrWhiteSpace(authInfo.Version))
{
authInfo.Version = device.AppVersion;
}
else if (!string.Equals(authInfo.Version, device.AppVersion, StringComparison.OrdinalIgnoreCase))
{
if (allowTokenInfoUpdate)
{
updateToken = true;
device.AppVersion = authInfo.Version;
}
}
if ((DateTime.UtcNow - device.DateLastActivity).TotalMinutes > 3)
{
device.DateLastActivity = DateTime.UtcNow;
updateToken = true;
}
authInfo.User = _userManager.GetUserById(device.UserId);
if (updateToken)
{
await _deviceManager.UpdateDevice(device).ConfigureAwait(false);
}
return authInfo;
}
var dbContext = await _jellyfinDbProvider.CreateDbContextAsync().ConfigureAwait(false);
await using (dbContext.ConfigureAwait(false))
{
var device = _deviceManager.GetDevices(
new DeviceQuery { AccessToken = token }).Items.FirstOrDefault();
if (device is not null)
var key = await dbContext.ApiKeys.FirstOrDefaultAsync(apiKey => apiKey.AccessToken == token).ConfigureAwait(false);
if (key is not null)
{
authInfo.IsAuthenticated = true;
var updateToken = false;
// TODO: Remove these checks for IsNullOrWhiteSpace
if (string.IsNullOrWhiteSpace(authInfo.Client))
{
authInfo.Client = device.AppName;
}
authInfo.Client = key.Name;
authInfo.Token = key.AccessToken;
if (string.IsNullOrWhiteSpace(authInfo.DeviceId))
{
authInfo.DeviceId = device.DeviceId;
authInfo.DeviceId = _serverApplicationHost.SystemId;
}
// Temporary. TODO - allow clients to specify that the token has been shared with a casting device
var allowTokenInfoUpdate = !authInfo.Client.Contains("chromecast", StringComparison.OrdinalIgnoreCase);
if (string.IsNullOrWhiteSpace(authInfo.Device))
{
authInfo.Device = device.DeviceName;
}
else if (!string.Equals(authInfo.Device, device.DeviceName, StringComparison.OrdinalIgnoreCase))
{
if (allowTokenInfoUpdate)
{
updateToken = true;
device.DeviceName = authInfo.Device;
}
authInfo.Device = _serverApplicationHost.Name;
}
if (string.IsNullOrWhiteSpace(authInfo.Version))
{
authInfo.Version = device.AppVersion;
}
else if (!string.Equals(authInfo.Version, device.AppVersion, StringComparison.OrdinalIgnoreCase))
{
if (allowTokenInfoUpdate)
{
updateToken = true;
device.AppVersion = authInfo.Version;
}
authInfo.Version = _serverApplicationHost.ApplicationVersionString;
}
if ((DateTime.UtcNow - device.DateLastActivity).TotalMinutes > 3)
{
device.DateLastActivity = DateTime.UtcNow;
updateToken = true;
}
authInfo.User = _userManager.GetUserById(device.UserId);
if (updateToken)
{
await _deviceManager.UpdateDevice(device).ConfigureAwait(false);
}
}
else
{
var key = await dbContext.ApiKeys.FirstOrDefaultAsync(apiKey => apiKey.AccessToken == token).ConfigureAwait(false);
if (key is not null)
{
authInfo.IsAuthenticated = true;
authInfo.Client = key.Name;
authInfo.Token = key.AccessToken;
if (string.IsNullOrWhiteSpace(authInfo.DeviceId))
{
authInfo.DeviceId = _serverApplicationHost.SystemId;
}
if (string.IsNullOrWhiteSpace(authInfo.Device))
{
authInfo.Device = _serverApplicationHost.Name;
}
if (string.IsNullOrWhiteSpace(authInfo.Version))
{
authInfo.Version = _serverApplicationHost.ApplicationVersionString;
}
authInfo.IsApiKey = true;
}
authInfo.IsApiKey = true;
}
return authInfo;
@@ -61,10 +61,10 @@ public sealed class DeviceAccessHost : IHostedService
private async Task UpdateDeviceAccess(User user)
{
var existing = _deviceManager.GetDevices(new DeviceQuery
var existing = (await _deviceManager.GetDevices(new DeviceQuery
{
UserId = user.Id
}).Items;
}).ConfigureAwait(false)).Items;
foreach (var device in existing)
{
+14
View File
@@ -2,12 +2,15 @@ using System;
using System.Collections.Generic;
using System.Reflection;
using Emby.Server.Implementations;
using Emby.Server.Implementations.MediaEncoding;
using Emby.Server.Implementations.ScheduledTasks;
using Emby.Server.Implementations.Session;
using Jellyfin.Api.WebSocketListeners;
using Jellyfin.Database.Implementations;
using Jellyfin.Drawing;
using Jellyfin.Drawing.Skia;
using Jellyfin.LiveTv;
using Jellyfin.Server.Extensions;
using Jellyfin.Server.Implementations.Activity;
using Jellyfin.Server.Implementations.Devices;
using Jellyfin.Server.Implementations.Events;
@@ -23,6 +26,7 @@ using MediaBrowser.Controller.Drawing;
using MediaBrowser.Controller.Events;
using MediaBrowser.Controller.Library;
using MediaBrowser.Controller.Lyrics;
using MediaBrowser.Controller.MediaEncoding;
using MediaBrowser.Controller.Net;
using MediaBrowser.Controller.Security;
using MediaBrowser.Controller.Trickplay;
@@ -39,6 +43,8 @@ namespace Jellyfin.Server
/// </summary>
public class CoreAppHost : ApplicationHost
{
private readonly IConfiguration _startupConfig;
/// <summary>
/// Initializes a new instance of the <see cref="CoreAppHost" /> class.
/// </summary>
@@ -57,6 +63,7 @@ namespace Jellyfin.Server
options,
startupConfig)
{
_startupConfig = startupConfig;
}
/// <inheritdoc/>
@@ -98,6 +105,13 @@ namespace Jellyfin.Server
serviceCollection.AddScoped<IAuthenticationManager, AuthenticationManager>();
// Transcode session store: Redis-backed when configured, no-op otherwise.
serviceCollection.AddTranscodeSessionStore(_startupConfig, Logger);
// Scan-leader lease: gates periodic library-mutating scheduled tasks to a single leader
// instance. Active by default once a Redis connection is configured, no-op otherwise.
serviceCollection.AddScanLeaderLease(_startupConfig, Logger);
foreach (var type in GetExportTypes<ILyricProvider>())
{
serviceCollection.AddSingleton(typeof(ILyricProvider), type);
@@ -0,0 +1,51 @@
using System.Collections.Generic;
using System.Linq;
using Microsoft.Extensions.Configuration;
namespace Jellyfin.Server.Extensions;
/// <summary>
/// Extensions for building the application configuration.
/// </summary>
public static class ConfigurationBuilderExtensions
{
/// <summary>
/// The environment variable prefix that maps onto this fork's own <c>Jellyfin:*</c> configuration
/// keys, for example <c>Jellyfin__TranscodeStore__RedisConnectionString</c>.
/// </summary>
public const string JellyfinSectionEnvironmentPrefix = "Jellyfin__";
/// <summary>
/// The configuration section root this fork keeps its own settings under.
/// </summary>
public const string JellyfinSectionRoot = "Jellyfin";
/// <summary>
/// Adds environment variables named <c>Jellyfin__Section__Key</c> as the configuration keys
/// <c>Jellyfin:Section:Key</c>.
/// </summary>
/// <remarks>
/// The base configuration only reads <c>JELLYFIN_</c> prefixed environment variables, so the
/// unprefixed form every manifest, chart and document uses would otherwise be dropped and the
/// feature it configures would stay off with no error.
/// </remarks>
/// <param name="builder">The configuration builder.</param>
/// <returns>The updated configuration builder.</returns>
public static IConfigurationBuilder AddJellyfinSectionEnvironmentVariables(this IConfigurationBuilder builder)
{
// Read through the framework provider so "__" to ":" normalisation and case handling match the
// prefixed form exactly; the prefix it strips is then put back as the section root.
var scoped = new ConfigurationBuilder()
.AddEnvironmentVariables(JellyfinSectionEnvironmentPrefix)
.Build();
var entries = scoped.AsEnumerable()
.Where(entry => entry.Value is not null)
.Select(entry => new KeyValuePair<string, string?>(
ConfigurationPath.Combine(JellyfinSectionRoot, entry.Key),
entry.Value))
.ToList();
return builder.AddInMemoryCollection(entries);
}
}
@@ -0,0 +1,71 @@
using System;
using Emby.Server.Implementations.ScheduledTasks;
using MediaBrowser.Controller.ScheduledTasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
namespace Jellyfin.Server.Extensions;
/// <summary>
/// Extensions for registering the scan-leader lease.
/// </summary>
public static class ScanLeaderServiceCollectionExtensions
{
private const string RedisConnectionStringKey = "Jellyfin:TranscodeStore:RedisConnectionString";
/// <summary>
/// Registers the scan-leader lease and reports at <see cref="LogLevel.Information"/> whether
/// timer-driven library tasks are gated to a single instance.
/// </summary>
/// <remarks>
/// Gating is on by default once a Redis connection string is configured: that is only set for a
/// multi-instance deployment, which is the only shape where running library scans on every
/// instance is wrong. Single-instance installs have no Redis and keep running every task locally.
/// </remarks>
/// <param name="serviceCollection">The service collection.</param>
/// <param name="configuration">The configuration to read <c>Jellyfin:ScanLeader</c> from.</param>
/// <param name="logger">The logger to report the gating decision on.</param>
/// <returns>The updated service collection.</returns>
public static IServiceCollection AddScanLeaderLease(
this IServiceCollection serviceCollection,
IConfiguration configuration,
ILogger logger)
{
ArgumentNullException.ThrowIfNull(configuration);
ArgumentNullException.ThrowIfNull(logger);
serviceCollection.Configure<ScanLeaderOptions>(configuration.GetSection(ScanLeaderOptions.ConfigurationSection));
var redisConfigured = !string.IsNullOrEmpty(configuration[RedisConnectionStringKey]);
var requested = bool.TryParse(configuration[ScanLeaderOptions.EnabledKey], out var explicitChoice)
? explicitChoice
: redisConfigured;
var active = requested && redisConfigured;
// The task worker reads Enabled from the bound options, so the effective decision has to land
// there as well as on the lease registration below.
serviceCollection.PostConfigure<ScanLeaderOptions>(options => options.Enabled = active);
if (active)
{
logger.LogInformation(
"Scan-leader gating is active: timer-driven library tasks run only on the instance holding the Redis scan-leader lease.");
return serviceCollection.AddSingleton<IScanLeaderLease, RedisScanLeaderLease>();
}
if (requested)
{
logger.LogWarning(
"Scan-leader gating is enabled but no Redis connection string is configured ({Key}), so it cannot run: timer-driven library tasks run on every instance.",
RedisConnectionStringKey);
}
else
{
logger.LogInformation("Scan-leader gating is off: timer-driven library tasks run on every instance.");
}
return serviceCollection.AddSingleton<IScanLeaderLease, NullScanLeaderLease>();
}
}
@@ -0,0 +1,87 @@
using System;
using System.Linq;
using Emby.Server.Implementations.MediaEncoding;
using MediaBrowser.Controller.MediaEncoding;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using StackExchange.Redis;
namespace Jellyfin.Server.Extensions;
/// <summary>
/// Extensions for registering the transcode session store.
/// </summary>
public static class TranscodeStoreServiceCollectionExtensions
{
/// <summary>
/// Registers the transcode session store, Redis-backed when a connection string is configured and
/// no-op otherwise, and reports the selected store at <see cref="LogLevel.Information"/>.
/// </summary>
/// <param name="serviceCollection">The service collection.</param>
/// <param name="configuration">The configuration to read <c>Jellyfin:TranscodeStore</c> from.</param>
/// <param name="logger">The logger to report the selected store on.</param>
/// <returns>The updated service collection.</returns>
public static IServiceCollection AddTranscodeSessionStore(
this IServiceCollection serviceCollection,
IConfiguration configuration,
ILogger logger)
{
ArgumentNullException.ThrowIfNull(configuration);
ArgumentNullException.ThrowIfNull(logger);
serviceCollection.Configure<TranscodeStoreOptions>(configuration.GetSection(TranscodeStoreOptions.ConfigurationSection));
var redisConnectionString = configuration[TranscodeStoreOptions.RedisConnectionStringKey];
if (string.IsNullOrEmpty(redisConnectionString))
{
logger.LogInformation(
"Transcode session store: {Store}. Cross-pod transcode takeover is off; set {Key} to enable it.",
nameof(NullTranscodeSessionStore),
TranscodeStoreOptions.RedisConnectionStringKey);
return serviceCollection.AddSingleton<ITranscodeSessionStore, NullTranscodeSessionStore>();
}
logger.LogInformation(
"Transcode session store: {Store} on {Endpoints}.",
nameof(RedisTranscodeSessionStore),
DescribeEndpoints(redisConnectionString));
serviceCollection.AddSingleton<IConnectionMultiplexer>(sp =>
{
try
{
return ConnectionMultiplexer.Connect(redisConnectionString);
}
catch (Exception ex)
{
sp.GetRequiredService<ILogger<CoreAppHost>>()
.LogError(ex, "Failed to connect to Redis. Check the {Key} configuration.", TranscodeStoreOptions.RedisConnectionStringKey);
throw;
}
});
serviceCollection.AddSingleton<ITranscodeSessionStore, RedisTranscodeSessionStore>();
serviceCollection.AddHostedService<TranscodeStoreConnectivityProbe>();
return serviceCollection;
}
/// <summary>
/// Renders the endpoints of a connection string for logging. The connection string itself is never
/// logged because it can carry a password.
/// </summary>
private static string DescribeEndpoints(string redisConnectionString)
{
try
{
return string.Join(
',',
ConfigurationOptions.Parse(redisConnectionString).EndPoints.Select(endpoint => endpoint.ToString()));
}
catch (ArgumentException)
{
return "(unparsable connection string)";
}
}
}
+1
View File
@@ -56,6 +56,7 @@
<PackageReference Include="Serilog.Sinks.Console" />
<PackageReference Include="Serilog.Sinks.File" />
<PackageReference Include="Serilog.Sinks.Graylog" />
<PackageReference Include="StackExchange.Redis" />
</ItemGroup>
<ItemGroup>
@@ -183,7 +183,13 @@ internal class JellyfinMigrationService
}
}
public async Task MigrateStepAsync(JellyfinMigrationStageTypes stage, IServiceProvider? serviceProvider)
/// <summary>
/// Runs all pending migrations of the requested stage.
/// </summary>
/// <param name="stage">The stage to migrate.</param>
/// <param name="serviceProvider">The service provider handed to the migrations.</param>
/// <returns>A value indicating whether at least one migration has been applied.</returns>
public async Task<bool> MigrateStepAsync(JellyfinMigrationStageTypes stage, IServiceProvider serviceProvider)
{
var logger = _startupLogger.With(_loggerFactory.CreateLogger<JellyfinMigrationService>()).BeginGroup($"Migrate stage {stage}.");
ICollection<CodeMigration> migrationStage = (Migrations.FirstOrDefault(e => e.Stage == stage) as ICollection<CodeMigration>) ?? [];
@@ -297,6 +303,8 @@ internal class JellyfinMigrationService
completedMigrations++;
}
return completedMigrations > 0;
}
}
@@ -445,10 +453,10 @@ internal class JellyfinMigrationService
private class InternalCodeMigration : IInternalMigration
{
private readonly CodeMigration _codeMigration;
private readonly IServiceProvider? _serviceProvider;
private readonly IServiceProvider _serviceProvider;
private JellyfinDbContext _dbContext;
public InternalCodeMigration(CodeMigration codeMigration, IServiceProvider? serviceProvider, JellyfinDbContext dbContext)
public InternalCodeMigration(CodeMigration codeMigration, IServiceProvider serviceProvider, JellyfinDbContext dbContext)
{
_codeMigration = codeMigration;
_serviceProvider = serviceProvider;
@@ -29,7 +29,7 @@ internal class MigrateLibraryUserData : IAsyncMigrationRoutine
private readonly IDbContextFactory<JellyfinDbContext> _provider;
public MigrateLibraryUserData(
IStartupLogger<MigrateLibraryDb> startupLogger,
IStartupLogger<MigrateLibraryUserData> startupLogger,
IDbContextFactory<JellyfinDbContext> provider,
IServerApplicationPaths paths)
{
@@ -24,7 +24,7 @@ internal class ReseedFolderFlag : IAsyncMigrationRoutine
private readonly IDbContextFactory<JellyfinDbContext> _provider;
public ReseedFolderFlag(
IStartupLogger<MigrateLibraryDb> startupLogger,
IStartupLogger<ReseedFolderFlag> startupLogger,
IDbContextFactory<JellyfinDbContext> provider,
IServerApplicationPaths paths)
{
@@ -36,7 +36,9 @@ internal class MigrateRatingLevels : IDatabaseMigrationRoutine
_logger.LogInformation("Recalculating parental rating levels based on rating string.");
using var context = _provider.CreateDbContext();
using var transaction = context.Database.BeginTransaction();
var ratings = context.BaseItems.AsNoTracking().Select(e => e.OfficialRating).Distinct();
// Read the whole list up front: the updates below run on the same connection, and a provider
// that cannot multiplex commands rejects them while the reader is still open.
var ratings = context.BaseItems.AsNoTracking().Select(e => e.OfficialRating).Distinct().ToList();
foreach (var rating in ratings)
{
if (string.IsNullOrEmpty(rating))
@@ -4,8 +4,6 @@ using System.Threading;
using System.Threading.Tasks;
using Jellyfin.Server.ServerSetupApp;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Logging;
namespace Jellyfin.Server.Migrations.Stages;
@@ -22,66 +20,45 @@ internal class CodeMigration(Type migrationType, JellyfinMigrationAttribute meta
return Metadata.Order.ToString("yyyyMMddHHmmsss", CultureInfo.InvariantCulture) + "_" + Metadata.Name!;
}
private IServiceCollection MigrationServices(IServiceProvider serviceProvider, IStartupLogger logger)
public async Task Perform(IServiceProvider serviceProvider, IStartupLogger logger, CancellationToken cancellationToken)
{
var childServiceCollection = new ServiceCollection()
.AddSingleton(serviceProvider)
.AddSingleton(logger)
.AddSingleton(typeof(IStartupLogger<>), typeof(NestedStartupLogger<>))
.AddSingleton<StartupLogTopic>(logger.Topic!);
foreach (ServiceDescriptor service in serviceProvider.GetRequiredService<IServiceCollection>())
{
if (service.Lifetime == ServiceLifetime.Singleton && !service.ServiceType.IsGenericTypeDefinition)
{
childServiceCollection.AddSingleton(service.ServiceType, _ => serviceProvider.GetService(service.ServiceType)!);
continue;
}
childServiceCollection.Add(service);
}
return childServiceCollection;
}
public async Task Perform(IServiceProvider? serviceProvider, IStartupLogger logger, CancellationToken cancellationToken)
{
#pragma warning disable CS0618 // Type or member is obsolete
if (typeof(IMigrationRoutine).IsAssignableFrom(MigrationType))
{
if (serviceProvider is null)
{
((IMigrationRoutine)Activator.CreateInstance(MigrationType)!).Perform();
}
else
{
using var migrationServices = MigrationServices(serviceProvider, logger).BuildServiceProvider();
((IMigrationRoutine)ActivatorUtilities.CreateInstance(migrationServices, MigrationType)).Perform();
#pragma warning restore CS0618 // Type or member is obsolete
}
}
else if (typeof(IAsyncMigrationRoutine).IsAssignableFrom(MigrationType))
{
if (serviceProvider is null)
{
await ((IAsyncMigrationRoutine)Activator.CreateInstance(MigrationType)!).PerformAsync(cancellationToken).ConfigureAwait(false);
}
else
{
using var migrationServices = MigrationServices(serviceProvider, logger).BuildServiceProvider();
await ((IAsyncMigrationRoutine)ActivatorUtilities.CreateInstance(migrationServices, MigrationType)).PerformAsync(cancellationToken).ConfigureAwait(false);
}
}
else
if (!IsMigrationRoutine(MigrationType))
{
throw new InvalidOperationException($"The type {MigrationType} does not implement either IMigrationRoutine or IAsyncMigrationRoutine and is not a valid migration type");
}
}
private class NestedStartupLogger<TCategory> : StartupLogger<TCategory>
{
public NestedStartupLogger(ILogger logger, StartupLogTopic topic) : base(logger, topic)
// The routine runs against a scope of the applications own container. Copying the application service
// descriptors into a child container instead would make that child container the owner of every singleton it
// forwards, so disposing it after the migration would also dispose the applications own instance of services
// like the ProviderManager and leave the server broken until the next restart.
var scope = serviceProvider.CreateAsyncScope();
await using (scope.ConfigureAwait(false))
{
// Nests everything the routine logs through an injected IStartupLogger under the migrations own topic.
using (StartupLogger.BeginAmbientTopic(logger.Topic))
{
await RunAsync(ActivatorUtilities.CreateInstance(scope.ServiceProvider, MigrationType), cancellationToken).ConfigureAwait(false);
}
}
}
// The obsolete IMigrationRoutine is still implemented by every routine that predates the async interface, so
// the members that have to touch it are grouped here behind a single suppression.
#pragma warning disable CS0618 // Type or member is obsolete
private static bool IsMigrationRoutine(Type migrationType)
{
return typeof(IMigrationRoutine).IsAssignableFrom(migrationType) || typeof(IAsyncMigrationRoutine).IsAssignableFrom(migrationType);
}
private static async Task RunAsync(object routine, CancellationToken cancellationToken)
{
if (routine is IMigrationRoutine migrationRoutine)
{
migrationRoutine.Perform();
return;
}
await ((IAsyncMigrationRoutine)routine).PerformAsync(cancellationToken).ConfigureAwait(false);
}
#pragma warning restore CS0618 // Type or member is obsolete
}
+42 -12
View File
@@ -24,6 +24,7 @@ using Jellyfin.Server.ServerSetupApp;
using MediaBrowser.Common.Configuration;
using MediaBrowser.Common.Net;
using MediaBrowser.Controller;
using MediaBrowser.Controller.Configuration;
using Microsoft.AspNetCore.Hosting;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
@@ -61,6 +62,7 @@ namespace Jellyfin.Server
private static ILogger _logger = NullLogger.Instance;
private static bool _restartOnShutdown;
private static IStartupLogger<JellyfinMigrationService>? _migrationLogger;
private static bool _optimizeDatabaseAfterMigration;
private static string? _restoreFromBackup;
/// <summary>
@@ -180,9 +182,7 @@ namespace Jellyfin.Server
})
.ConfigureAppConfiguration(config => config.ConfigureAppConfiguration(options, appPaths, startupConfig))
.UseSerilog()
.ConfigureServices(e => e
.RegisterStartupLogger()
.AddSingleton<IServiceCollection>(e))
.ConfigureServices(e => e.RegisterStartupLogger())
.Build();
/*
@@ -209,14 +209,15 @@ namespace Jellyfin.Server
await jellyfinMigrationService.PrepareSystemForMigration(_logger).ConfigureAwait(false);
// "Preparing migrations" carries through the DB read; per-migration progress is reported
// as "Running migration X of Y" from inside the step once the pending set is known.
await jellyfinMigrationService.MigrateStepAsync(JellyfinMigrationStageTypes.CoreInitialisation, appHost.ServiceProvider).ConfigureAwait(false);
_optimizeDatabaseAfterMigration |= await jellyfinMigrationService.MigrateStepAsync(JellyfinMigrationStageTypes.CoreInitialisation, appHost.ServiceProvider).ConfigureAwait(false);
SetupServer.ReportActivity(StartupActivity.InitializingServices);
await appHost.InitializeServices(startupConfig).ConfigureAwait(false);
_appHost = appHost;
await jellyfinMigrationService.MigrateStepAsync(JellyfinMigrationStageTypes.AppInitialisation, appHost.ServiceProvider).ConfigureAwait(false);
_optimizeDatabaseAfterMigration |= await jellyfinMigrationService.MigrateStepAsync(JellyfinMigrationStageTypes.AppInitialisation, appHost.ServiceProvider).ConfigureAwait(false);
await jellyfinMigrationService.CleanupSystemAfterMigration(_logger).ConfigureAwait(false);
await OptimizeDatabaseAfterMigrationAsync(appHost.ServiceProvider).ConfigureAwait(false);
try
{
configurationCompleted = true;
@@ -271,12 +272,11 @@ namespace Jellyfin.Server
// Don't throw additional exception if startup failed.
if (appHost.ServiceProvider is not null)
{
_logger.LogInformation("Running query planner optimizations in the database... This might take a while");
_logger.LogInformation("Optimizing the database... This might take a while");
// Deliberately untimed: a truncated optimization leaves the statistics incomplete.
var databaseProvider = appHost.ServiceProvider.GetRequiredService<IJellyfinDatabaseProvider>();
using var shutdownSource = new CancellationTokenSource();
shutdownSource.CancelAfter((int)TimeSpan.FromSeconds(60).TotalMicroseconds);
await databaseProvider.RunShutdownTask(shutdownSource.Token).ConfigureAwait(false);
await databaseProvider.RunShutdownTask(CancellationToken.None).ConfigureAwait(false);
}
_appHost = null;
@@ -305,16 +305,19 @@ namespace Jellyfin.Server
.AddJellyfinDbContext(startupConfigurationManager, startupConfig)
.AddSingleton<IApplicationPaths>(appPaths)
.AddSingleton<ServerApplicationPaths>(appPaths)
// Required by the NpgsqlDataSource factory in AddJellyfinDbContext when
// DatabaseType=Jellyfin-PostgreSQL — the factory resolves this from DI
// to read CustomProviderOptions and pool settings.
.AddSingleton<IServerConfigurationManager>(startupConfigurationManager)
.RegisterStartupLogger();
migrationStartupServiceProvider.AddSingleton(migrationStartupServiceProvider);
var startupService = migrationStartupServiceProvider.BuildServiceProvider();
PrepareDatabaseProvider(startupService);
var jellyfinMigrationService = ActivatorUtilities.CreateInstance<JellyfinMigrationService>(startupService);
await jellyfinMigrationService.CheckFirstTimeRunOrMigration(appPaths, startupOptions).ConfigureAwait(false);
await jellyfinMigrationService.MigrateStepAsync(Migrations.Stages.JellyfinMigrationStageTypes.PreInitialisation, startupService).ConfigureAwait(false);
_optimizeDatabaseAfterMigration |= await jellyfinMigrationService.MigrateStepAsync(Migrations.Stages.JellyfinMigrationStageTypes.PreInitialisation, startupService).ConfigureAwait(false);
}
/// <summary>
@@ -329,7 +332,32 @@ namespace Jellyfin.Server
public static async Task ApplyCoreMigrationsAsync(IServiceProvider serviceProvider, Migrations.Stages.JellyfinMigrationStageTypes jellyfinMigrationStage)
{
var jellyfinMigrationService = ActivatorUtilities.CreateInstance<JellyfinMigrationService>(serviceProvider, _migrationLogger!);
await jellyfinMigrationService.MigrateStepAsync(jellyfinMigrationStage, serviceProvider).ConfigureAwait(false);
_optimizeDatabaseAfterMigration |= await jellyfinMigrationService.MigrateStepAsync(jellyfinMigrationStage, serviceProvider).ConfigureAwait(false);
}
private static async Task OptimizeDatabaseAfterMigrationAsync(IServiceProvider serviceProvider)
{
if (!_optimizeDatabaseAfterMigration)
{
return;
}
// Reset first: a restart runs no migrations and must not optimize again.
_optimizeDatabaseAfterMigration = false;
SetupServer.ReportActivity(StartupActivity.OptimizingDatabase);
_logger.LogInformation("Migrations have been applied, optimizing the database... This might take a while");
try
{
// Deliberately untimed: incomplete statistics are worse than a slow start.
var databaseProvider = serviceProvider.GetRequiredService<IJellyfinDatabaseProvider>();
await databaseProvider.RunScheduledOptimisation(CancellationToken.None).ConfigureAwait(false);
}
catch (Exception ex)
{
// A missed optimization only costs performance, so never fail startup over this.
_logger.LogError(ex, "Error while optimizing the database after migration");
}
}
/// <summary>
@@ -363,6 +391,8 @@ namespace Jellyfin.Server
.AddInMemoryCollection(inMemoryDefaultConfig)
.AddJsonFile(LoggingConfigFileDefault, optional: false, reloadOnChange: true)
.AddJsonFile(LoggingConfigFileSystem, optional: true, reloadOnChange: true)
// Added before the prefixed source so an explicit JELLYFIN_ variable still wins.
.AddJellyfinSectionEnvironmentVariables()
.AddEnvironmentVariables("JELLYFIN_")
.AddInMemoryCollection(commandLineOpts.ConvertToConfig());
}
@@ -27,6 +27,9 @@ public static class StartupActivity
/// <summary>Bringing up core services and plugins.</summary>
public const string InitializingServices = "Initializing services";
/// <summary>Refreshing the database statistics after migrations have run.</summary>
public const string OptimizingDatabase = "Optimizing database";
/// <summary>Running the final startup tasks.</summary>
public const string FinishingStartup = "Finishing startup";
@@ -1,5 +1,6 @@
using System;
using System.Globalization;
using System.Threading;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
@@ -8,6 +9,8 @@ namespace Jellyfin.Server.ServerSetupApp;
/// <inheritdoc/>
public class StartupLogger : IStartupLogger
{
private static readonly AsyncLocal<StartupLogTopic?> _ambientTopic = new();
private readonly StartupLogTopic? _topic;
/// <summary>
@@ -17,6 +20,7 @@ public class StartupLogger : IStartupLogger
public StartupLogger(ILogger logger)
{
BaseLogger = logger;
_topic = _ambientTopic.Value;
}
/// <summary>
@@ -39,6 +43,18 @@ public class StartupLogger : IStartupLogger
/// </summary>
protected ILogger BaseLogger { get; set; }
/// <summary>
/// Makes <paramref name="topic"/> the topic that loggers created on this execution context attach to.
/// </summary>
/// <param name="topic">The topic to nest newly created loggers under.</param>
/// <returns>A scope that restores the previously ambient topic when disposed.</returns>
internal static IDisposable BeginAmbientTopic(StartupLogTopic? topic)
{
var scope = new AmbientTopicScope(_ambientTopic.Value);
_ambientTopic.Value = topic;
return scope;
}
/// <inheritdoc/>
public IStartupLogger BeginGroup(FormattableString logEntry)
{
@@ -121,4 +137,19 @@ public class StartupLogger : IStartupLogger
Topic.Children.Add(startupEntry);
}
}
private sealed class AmbientTopicScope : IDisposable
{
private readonly StartupLogTopic? _previous;
public AmbientTopicScope(StartupLogTopic? previous)
{
_previous = previous;
}
public void Dispose()
{
_ambientTopic.Value = _previous;
}
}
}
+23
View File
@@ -68,6 +68,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.Server.Tests", "te
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.Server.Integration.Tests", "tests\Jellyfin.Server.Integration.Tests\Jellyfin.Server.Integration.Tests.csproj", "{68B0B823-A5AC-4E8B-82EA-965AAC7BF76E}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Jellyfin.Database.Tests.PostgreSQL", "tests\Jellyfin.Database.Tests.PostgreSQL\Jellyfin.Database.Tests.PostgreSQL.csproj", "{B5F2C3A1-9D4E-4F8A-B123-456789ABCDEF}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.Providers.Tests", "tests\Jellyfin.Providers.Tests\Jellyfin.Providers.Tests.csproj", "{A964008C-2136-4716-B6CB-B3426C22320A}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{C9F0AB5D-F4D7-40C8-A353-3305C86D6D4C}"
@@ -95,10 +97,16 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Jellyfin.Database", "Jellyf
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.Database.Providers.Sqlite", "src\Jellyfin.Database\Jellyfin.Database.Providers.Sqlite\Jellyfin.Database.Providers.Sqlite.csproj", "{A5590358-33CC-4B39-BDE7-DC62FEB03C76}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.Database.Providers.PostgreSQL", "src\Jellyfin.Database\Jellyfin.Database.Providers.PostgreSQL\Jellyfin.Database.Providers.PostgreSQL.csproj", "{B3C4D5E6-F7A8-4B9C-0D1E-2F3A4B5C6D7E}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.Database.Implementations", "src\Jellyfin.Database\Jellyfin.Database.Implementations\Jellyfin.Database.Implementations.csproj", "{8C9F9221-8415-496C-B1F5-E7756F03FA59}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.CodeAnalysis", "src\Jellyfin.CodeAnalysis\Jellyfin.CodeAnalysis.csproj", "{11643D0F-6761-4EF7-AB71-6F9F8DE00714}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "tools", "tools", "{3C85DA50-31AC-40D3-BCF4-F1B14C420996}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Jellyfin.DbMigrator", "tools\Jellyfin.DbMigrator\Jellyfin.DbMigrator.csproj", "{6F7187CB-E1CB-4583-98CF-0FB87F21E844}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Jellyfin.Drawing.Skia.Tests", "tests\Jellyfin.Drawing.Skia.Tests\Jellyfin.Drawing.Skia.Tests.csproj", "{E24A279C-9A37-419A-8F9C-853C11FBE753}"
EndProject
Global
@@ -219,6 +227,10 @@ Global
{68B0B823-A5AC-4E8B-82EA-965AAC7BF76E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{68B0B823-A5AC-4E8B-82EA-965AAC7BF76E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{68B0B823-A5AC-4E8B-82EA-965AAC7BF76E}.Release|Any CPU.Build.0 = Release|Any CPU
{B5F2C3A1-9D4E-4F8A-B123-456789ABCDEF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{B5F2C3A1-9D4E-4F8A-B123-456789ABCDEF}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B5F2C3A1-9D4E-4F8A-B123-456789ABCDEF}.Release|Any CPU.ActiveCfg = Release|Any CPU
{B5F2C3A1-9D4E-4F8A-B123-456789ABCDEF}.Release|Any CPU.Build.0 = Release|Any CPU
{A964008C-2136-4716-B6CB-B3426C22320A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{A964008C-2136-4716-B6CB-B3426C22320A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{A964008C-2136-4716-B6CB-B3426C22320A}.Release|Any CPU.ActiveCfg = Release|Any CPU
@@ -259,6 +271,10 @@ Global
{A5590358-33CC-4B39-BDE7-DC62FEB03C76}.Debug|Any CPU.Build.0 = Debug|Any CPU
{A5590358-33CC-4B39-BDE7-DC62FEB03C76}.Release|Any CPU.ActiveCfg = Release|Any CPU
{A5590358-33CC-4B39-BDE7-DC62FEB03C76}.Release|Any CPU.Build.0 = Release|Any CPU
{B3C4D5E6-F7A8-4B9C-0D1E-2F3A4B5C6D7E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{B3C4D5E6-F7A8-4B9C-0D1E-2F3A4B5C6D7E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B3C4D5E6-F7A8-4B9C-0D1E-2F3A4B5C6D7E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{B3C4D5E6-F7A8-4B9C-0D1E-2F3A4B5C6D7E}.Release|Any CPU.Build.0 = Release|Any CPU
{8C9F9221-8415-496C-B1F5-E7756F03FA59}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8C9F9221-8415-496C-B1F5-E7756F03FA59}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8C9F9221-8415-496C-B1F5-E7756F03FA59}.Release|Any CPU.ActiveCfg = Release|Any CPU
@@ -267,6 +283,10 @@ Global
{11643D0F-6761-4EF7-AB71-6F9F8DE00714}.Debug|Any CPU.Build.0 = Debug|Any CPU
{11643D0F-6761-4EF7-AB71-6F9F8DE00714}.Release|Any CPU.ActiveCfg = Release|Any CPU
{11643D0F-6761-4EF7-AB71-6F9F8DE00714}.Release|Any CPU.Build.0 = Release|Any CPU
{6F7187CB-E1CB-4583-98CF-0FB87F21E844}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{6F7187CB-E1CB-4583-98CF-0FB87F21E844}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6F7187CB-E1CB-4583-98CF-0FB87F21E844}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6F7187CB-E1CB-4583-98CF-0FB87F21E844}.Release|Any CPU.Build.0 = Release|Any CPU
{E24A279C-9A37-419A-8F9C-853C11FBE753}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{E24A279C-9A37-419A-8F9C-853C11FBE753}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E24A279C-9A37-419A-8F9C-853C11FBE753}.Release|Any CPU.ActiveCfg = Release|Any CPU
@@ -290,6 +310,7 @@ Global
{42816EA8-4511-4CBF-A9C7-7791D5DDDAE6} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
{3ADBCD8C-C0F2-4956-8FDC-35D686B74CF9} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
{68B0B823-A5AC-4E8B-82EA-965AAC7BF76E} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
{B5F2C3A1-9D4E-4F8A-B123-456789ABCDEF} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
{A964008C-2136-4716-B6CB-B3426C22320A} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
{750B8757-BE3D-4F8C-941A-FBAD94904ADA} = {C9F0AB5D-F4D7-40C8-A353-3305C86D6D4C}
{332A5C7A-F907-47CA-910E-BE6F7371B9E0} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
@@ -301,8 +322,10 @@ Global
{8C6B2B13-58A4-4506-9DAB-1F882A093FE0} = {C9F0AB5D-F4D7-40C8-A353-3305C86D6D4C}
{4C54CE05-69C8-48FA-8785-39F7F6DB1CAD} = {C9F0AB5D-F4D7-40C8-A353-3305C86D6D4C}
{A5590358-33CC-4B39-BDE7-DC62FEB03C76} = {4C54CE05-69C8-48FA-8785-39F7F6DB1CAD}
{B3C4D5E6-F7A8-4B9C-0D1E-2F3A4B5C6D7E} = {4C54CE05-69C8-48FA-8785-39F7F6DB1CAD}
{8C9F9221-8415-496C-B1F5-E7756F03FA59} = {4C54CE05-69C8-48FA-8785-39F7F6DB1CAD}
{11643D0F-6761-4EF7-AB71-6F9F8DE00714} = {C9F0AB5D-F4D7-40C8-A353-3305C86D6D4C}
{6F7187CB-E1CB-4583-98CF-0FB87F21E844} = {3C85DA50-31AC-40D3-BCF4-F1B14C420996}
{E24A279C-9A37-419A-8F9C-853C11FBE753} = {FBBB5129-006E-4AD7-BAD5-8B7CA1D10ED6}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
@@ -47,29 +47,29 @@ public interface IDeviceManager
/// Gets the device information.
/// </summary>
/// <param name="id">The identifier.</param>
/// <returns>DeviceInfoDto.</returns>
DeviceInfoDto? GetDevice(string id);
/// <returns>A <see cref="Task"/> representing the retrieval of the device information.</returns>
Task<DeviceInfoDto?> GetDevice(string id);
/// <summary>
/// Gets devices based on the provided query.
/// </summary>
/// <param name="query">The device query.</param>
/// <returns>A <see cref="Task{QueryResult}"/> representing the retrieval of the devices.</returns>
QueryResult<Device> GetDevices(DeviceQuery query);
Task<QueryResult<Device>> GetDevices(DeviceQuery query);
/// <summary>
/// Gets device information based on the provided query.
/// </summary>
/// <param name="query">The device query.</param>
/// <returns>A <see cref="Task{QueryResult}"/> representing the retrieval of the device information.</returns>
QueryResult<DeviceInfo> GetDeviceInfos(DeviceQuery query);
Task<QueryResult<DeviceInfo>> GetDeviceInfos(DeviceQuery query);
/// <summary>
/// Gets the device information.
/// </summary>
/// <param name="userId">The user's id, or <c>null</c>.</param>
/// <returns>IEnumerable&lt;DeviceInfoDto&gt;.</returns>
QueryResult<DeviceInfoDto> GetDevicesForUser(Guid? userId);
/// <returns>A <see cref="Task{QueryResult}"/> representing the retrieval of the device information.</returns>
Task<QueryResult<DeviceInfoDto>> GetDevicesForUser(Guid? userId);
/// <summary>
/// Deletes a device.
@@ -105,8 +105,8 @@ public interface IDeviceManager
/// Gets the options of a device.
/// </summary>
/// <param name="deviceId">The device id.</param>
/// <returns><see cref="DeviceOptions"/> of the device.</returns>
DeviceOptionsDto? GetDeviceOptions(string deviceId);
/// <returns>A <see cref="Task"/> representing the retrieval of the <see cref="DeviceOptions"/> of the device.</returns>
Task<DeviceOptionsDto?> GetDeviceOptions(string deviceId);
/// <summary>
/// Gets the dto for client capabilities.
@@ -432,12 +432,18 @@ namespace MediaBrowser.Controller.Entities
public string? HasNoSubtitleTrackWithLanguage { get; set; }
/// <summary>
/// Gets or sets a value indicating whether to return only items nothing names any more.
/// </summary>
public bool? IsDeadArtist { get; set; }
public bool? IsDeadStudio { get; set; }
public bool? IsDeadGenre { get; set; }
/// <summary>
/// Gets or sets a value indicating whether to return only items nothing names any more.
/// </summary>
public bool? IsDeadPerson { get; set; }
/// <summary>
@@ -106,6 +106,13 @@ namespace MediaBrowser.Controller.Library
/// <returns>Task{Person}.</returns>
Person? GetPerson(string name);
/// <summary>
/// Gets a Person, creating and persisting it if no item exists for the name yet.
/// </summary>
/// <param name="name">The name of the person.</param>
/// <returns>The person.</returns>
Person GetOrCreatePerson(string name);
/// <summary>
/// Finds the by path.
/// </summary>
@@ -152,15 +159,6 @@ namespace MediaBrowser.Controller.Library
/// <exception cref="ArgumentOutOfRangeException">Throws if year is invalid.</exception>
Year GetYear(int value);
/// <summary>
/// Validate and refresh the People sub-set of the IBN.
/// The items are stored in the db but not loaded into memory until actually requested by an operation.
/// </summary>
/// <param name="progress">The progress.</param>
/// <param name="cancellationToken">The cancellation token.</param>
/// <returns>Task.</returns>
Task ValidatePeopleAsync(IProgress<double> progress, CancellationToken cancellationToken);
/// <summary>
/// Reloads the root media folder.
/// </summary>
@@ -709,6 +707,14 @@ namespace MediaBrowser.Controller.Library
/// <returns><c>true</c> if ignored, <c>false</c> otherwise.</returns>
bool IgnoreFile(FileSystemMetadata file, BaseItem parent);
/// <summary>
/// Gets the id a <see cref="Person"/> item for the name would have, without looking it up
/// or creating it.
/// </summary>
/// <param name="name">The name of the person.</param>
/// <returns>The item id for the name.</returns>
Guid GetPersonId(string name);
Guid GetStudioId(string name);
Guid GetGenreId(string name);
@@ -753,6 +759,18 @@ namespace MediaBrowser.Controller.Library
/// <returns>The item counts grouped by type.</returns>
ItemCounts GetItemCountsForNameItem(BaseItemKind kind, Guid id, BaseItemKind[] relatedItemKinds, User? user);
/// <summary>
/// Gets item counts for several "by-name" items of the same kind. Kinds keyed by a cleaned
/// item value - artists, genres and studios - are answered in one set of queries for the
/// whole batch; the rest fall back to one query per item.
/// </summary>
/// <param name="kind">The kind of the name items.</param>
/// <param name="ids">The IDs of the name items.</param>
/// <param name="relatedItemKinds">The item kinds to count.</param>
/// <param name="user">The user for access filtering.</param>
/// <returns>The item counts of each requested id.</returns>
Dictionary<Guid, ItemCounts> GetItemCountsForNameItems(BaseItemKind kind, IReadOnlyList<Guid> ids, BaseItemKind[] relatedItemKinds, User? user);
/// <summary>
/// Batch-fetches child counts for multiple parent folders.
/// Returns the count of immediate children (non-recursive) for each parent.
@@ -17,7 +17,8 @@ namespace MediaBrowser.Controller.LibraryTaskScheduler;
/// </summary>
public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibraryScheduler, IAsyncDisposable
{
private const int CleanupGracePeriod = 60;
private static readonly TimeSpan _cleanupGracePeriod = TimeSpan.FromSeconds(60);
private readonly IHostApplicationLifetime _hostApplicationLifetime;
private readonly ILogger<LimitedConcurrencyLibraryScheduler> _logger;
private readonly IServerConfigurationManager _serverConfigurationManager;
@@ -31,6 +32,8 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
private readonly Lock _taskLock = new();
private readonly Channel<TaskQueueItem> _tasks = Channel.CreateUnbounded<TaskQueueItem>();
private readonly CancellationTokenSource _disposeTokenSource = new();
private readonly TimeSpan _gracePeriod;
private volatile int _workCounter;
private Task? _cleanupTask;
@@ -46,10 +49,34 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
IHostApplicationLifetime hostApplicationLifetime,
ILogger<LimitedConcurrencyLibraryScheduler> logger,
IServerConfigurationManager serverConfigurationManager)
: this(hostApplicationLifetime, logger, serverConfigurationManager, _cleanupGracePeriod)
{
}
internal LimitedConcurrencyLibraryScheduler(
IHostApplicationLifetime hostApplicationLifetime,
ILogger<LimitedConcurrencyLibraryScheduler> logger,
IServerConfigurationManager serverConfigurationManager,
TimeSpan gracePeriod)
{
_hostApplicationLifetime = hostApplicationLifetime;
_logger = logger;
_serverConfigurationManager = serverConfigurationManager;
_gracePeriod = gracePeriod;
}
/// <summary>
/// Gets the number of runners the scheduler currently keeps alive.
/// </summary>
internal int ActiveRunnerCount
{
get
{
lock (_taskLock)
{
return _taskRunners.Count;
}
}
}
private void ScheduleTaskCleanup()
@@ -68,31 +95,65 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
async Task RunCleanupTask()
{
_logger.LogDebug("Schedule cleanup task in {CleanupGracePerioid} sec.", CleanupGracePeriod);
await Task.Delay(TimeSpan.FromSeconds(CleanupGracePeriod)).ConfigureAwait(false);
if (_disposed)
while (true)
{
_logger.LogDebug("Abort cleaning up, already disposed.");
return;
}
lock (_taskLock)
{
if (_tasks.Reader.Count > 0 || _workCounter > 0)
_logger.LogDebug("Schedule cleanup task in {CleanupGracePeriod}.", _gracePeriod);
try
{
_logger.LogDebug("Delay cleanup task, operations still running.");
// tasks are still there so its still in use. Reschedule cleanup task.
// we cannot just exit here and rely on the other invoker because there is a considerable timeframe where it could have already ended.
_cleanupTask = RunCleanupTask();
await Task.Delay(_gracePeriod, _disposeTokenSource.Token).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
_logger.LogDebug("Abort cleaning up, already disposed.");
return;
}
}
_logger.LogDebug("Cleanup runners.");
foreach (var item in _taskRunners.ToArray())
if (_disposed)
{
_logger.LogDebug("Abort cleaning up, already disposed.");
return;
}
CancellationTokenSource[] runners;
lock (_taskLock)
{
if (_tasks.Reader.Count > 0 || _workCounter > 0)
{
_logger.LogDebug("Delay cleanup task, operations still running.");
// tasks are still there so its still in use. Wait another grace period.
// we cannot just exit here and rely on the other invoker because there is a considerable timeframe where it could have already ended.
continue;
}
runners = [.. _taskRunners.Keys];
// Retire the runners before they are told to stop: an operation starting while
// they wind down must spawn its own instead of counting these towards the fanout.
_taskRunners.Clear();
// Hand the next operation the ability to schedule a cleanup again. Without this
// the very first cleanup would be the only one that ever runs.
_cleanupTask = null;
}
_logger.LogDebug("Cleanup runners.");
await StopRunners(runners).ConfigureAwait(false);
return;
}
}
}
private static async Task StopRunners(CancellationTokenSource[] runners)
{
foreach (var runner in runners)
{
try
{
await item.Key.CancelAsync().ConfigureAwait(false);
_taskRunners.Remove(item.Key);
await runner.CancelAsync().ConfigureAwait(false);
}
catch (ObjectDisposedException)
{
// The runner already stopped on its own and disposed its stop source.
}
}
}
@@ -127,12 +188,17 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
{
var stopToken = new CancellationTokenSource();
var combinedSource = CancellationTokenSource.CreateLinkedTokenSource(stopToken.Token, _hostApplicationLifetime.ApplicationStopping);
// Keyed on its own stop source, because cancelling that is what reaches the linked
// source the runner waits on. Cancellation does not travel the other way.
// Started without the runner's own token: a task cancelled before it is scheduled
// never runs its body, so it would never take itself out of _taskRunners again.
_taskRunners.Add(
combinedSource,
stopToken,
Task.Factory.StartNew(
ItemWorker,
(combinedSource, stopToken),
combinedSource.Token,
(stopToken, combinedSource),
CancellationToken.None,
TaskCreationOptions.PreferFairness,
TaskScheduler.Default));
}
@@ -145,7 +211,7 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
_deadlockDetector.Value = stopToken.TaskStop;
try
{
while (!stopToken.GlobalStop.Token.IsCancellationRequested)
while (!stopToken.GlobalStop.IsCancellationRequested)
{
var item = await _tasks.Reader.ReadAsync(stopToken.GlobalStop.Token).ConfigureAwait(false);
try
@@ -162,15 +228,24 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
}
}
}
catch (OperationCanceledException) when (stopToken.TaskStop.IsCancellationRequested)
catch (OperationCanceledException) when (stopToken.GlobalStop.IsCancellationRequested)
{
// thats how you do it, interupt the waiter thread. There is nothing to do here when it was on purpose.
}
catch (ChannelClosedException)
{
// the scheduler was disposed and will not hand out any more work.
}
finally
{
_logger.LogDebug("Cleanup Runner'.");
_deadlockDetector.Value = default!;
_taskRunners.Remove(stopToken.TaskStop);
lock (_taskLock)
{
_taskRunners.Remove(stopToken.TaskStop);
}
stopToken.GlobalStop.Dispose();
stopToken.TaskStop.Dispose();
}
@@ -195,7 +270,7 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
finally
{
item.Progress.Report(100);
item.Done.SetResult();
item.Done.TrySetResult();
}
}
@@ -285,16 +360,33 @@ public sealed class LimitedConcurrencyLibraryScheduler : ILimitedConcurrencyLibr
_disposed = true;
_tasks.Writer.Complete();
foreach (var item in _taskRunners)
// Nobody is left to run these, so release whoever is waiting on them.
while (_tasks.Reader.TryRead(out var item))
{
await item.Key.CancelAsync().ConfigureAwait(false);
item.Done.TrySetResult();
}
if (_cleanupTask is not null)
CancellationTokenSource[] runners;
Task? cleanupTask;
lock (_taskLock)
{
await _cleanupTask.ConfigureAwait(false);
_cleanupTask?.Dispose();
runners = [.. _taskRunners.Keys];
_taskRunners.Clear();
cleanupTask = _cleanupTask;
}
await StopRunners(runners).ConfigureAwait(false);
// Cuts the grace period short instead of holding up shutdown for the rest of it.
await _disposeTokenSource.CancelAsync().ConfigureAwait(false);
if (cleanupTask is not null)
{
await cleanupTask.ConfigureAwait(false);
}
_disposeTokenSource.Dispose();
}
private class TaskQueueItem
@@ -18,7 +18,6 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BitFaster.Caching" />
<PackageReference Include="Microsoft.Extensions.Configuration.Binder" />
</ItemGroup>
@@ -442,7 +442,8 @@ namespace MediaBrowser.Controller.MediaEncoding
&& (state.VideoStream.VideoRangeType == VideoRangeType.HDR10
|| IsHdr10Plus(state.VideoStream)
|| IsDoviWithHdr10Bl(state.VideoStream)
|| state.VideoStream.VideoRangeType == VideoRangeType.HLG);
|| state.VideoStream.VideoRangeType == VideoRangeType.HLG
|| state.VideoStream.VideoRangeType == VideoRangeType.DOVIInvalid);
}
private static bool IsDeinterlaceAvailable(EncodingJobInfo state)
@@ -695,7 +696,11 @@ namespace MediaBrowser.Controller.MediaEncoding
"ogg" or "oga" or "ogv" or "webm" or "webma" => "opus",
"m4a" or "m4b" or "mp4" or "mov" or "mkv" or "mka" => "aac",
"ts" or "avi" or "flv" or "f4v" or "swf" => "mp3",
_ => inferredCodec
// Containers that share their name with the codec they carry.
"aac" or "ac3" or "alac" or "dts" or "eac3" or "flac" or "mp2" or "mp3" or "opus" or "truehd" or "vorbis" => inferredCodec,
// Anything else - manifests such as m3u8/mpd in particular - names a container that
// is not an audio codec. Never hand that name to ffmpeg as an encoder.
_ => "aac"
};
}
@@ -1386,7 +1391,8 @@ namespace MediaBrowser.Controller.MediaEncoding
or VideoRangeType.DOVIWithEL
or VideoRangeType.DOVIWithHDR10Plus
or VideoRangeType.DOVIWithELHDR10Plus
or VideoRangeType.DOVIInvalid;
|| (rangeType == VideoRangeType.DOVIInvalid
&& string.Equals(stream.ColorTransfer, "smpte2084", StringComparison.OrdinalIgnoreCase)); // invalid may be hlg now
}
public static bool IsDovi(MediaStream stream)
@@ -1396,7 +1402,8 @@ namespace MediaBrowser.Controller.MediaEncoding
return IsDoviWithHdr10Bl(stream)
|| (rangeType is VideoRangeType.DOVI
or VideoRangeType.DOVIWithHLG
or VideoRangeType.DOVIWithSDR);
or VideoRangeType.DOVIWithSDR
or VideoRangeType.DOVIInvalid);
}
public static bool IsHdr10Plus(MediaStream stream)
@@ -1416,7 +1423,8 @@ namespace MediaBrowser.Controller.MediaEncoding
private static DynamicHdrMetadataRemovalPlan ShouldRemoveDynamicHdrMetadata(EncodingJobInfo state)
{
var videoStream = state.VideoStream;
if (videoStream.VideoRange is not VideoRange.HDR)
if (videoStream.VideoRange is not VideoRange.HDR
&& videoStream.VideoRangeType != VideoRangeType.DOVIInvalid)
{
return DynamicHdrMetadataRemovalPlan.None;
}
@@ -6311,7 +6319,7 @@ namespace MediaBrowser.Controller.MediaEncoding
string.Join(',', overlayFilters));
var mapPrefix = Convert.ToInt32(state.SubtitleStream.IsExternal);
var subtitleStreamIndex = FindIndex(state.MediaSource.MediaStreams, state.SubtitleStream);
var subtitleStreamIndex = GetSubtitleStreamIndexForFfmpeg(state.MediaSource, state.SubtitleStream);
var videoStreamIndex = FindIndex(state.MediaSource.MediaStreams, state.VideoStream);
if (hasSubs)
@@ -7943,6 +7951,24 @@ namespace MediaBrowser.Controller.MediaEncoding
return -1;
}
public static int GetSubtitleStreamIndexForFfmpeg(MediaSourceInfo mediaSource, MediaStream subtitleStream)
{
var index = FindIndex(mediaSource.MediaStreams, subtitleStream);
if (index == -1 || subtitleStream.IsExternal || mediaSource.VideoType != VideoType.BluRay)
{
return index;
}
var hiddenStreamsBefore = mediaSource.MediaStreams.Count(s =>
s.Type == MediaStreamType.Audio
&& !s.IsExternal
&& (string.Equals(s.Codec, "truehd", StringComparison.OrdinalIgnoreCase)
|| string.Equals(s.Codec, "atmos", StringComparison.OrdinalIgnoreCase))
&& s.Index < subtitleStream.Index);
return index + hiddenStreamsBefore;
}
public static bool IsCopyCodec(string codec)
{
return string.Equals(codec, "copy", StringComparison.OrdinalIgnoreCase);
@@ -0,0 +1,79 @@
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace MediaBrowser.Controller.MediaEncoding;
/// <summary>
/// Provides a durable store for HLS transcoding session state, enabling
/// HA recovery and lease-based ownership between pods.
/// </summary>
public interface ITranscodeSessionStore
{
/// <summary>
/// Attempts to retrieve a transcoding session by its play session identifier.
/// </summary>
/// <param name="playSessionId">The play session identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>
/// The <see cref="TranscodeSession"/> if it exists and its lease has not expired;
/// otherwise <c>null</c>.
/// </returns>
Task<TranscodeSession?> TryGetAsync(string playSessionId, CancellationToken cancellationToken = default);
/// <summary>
/// Attempts to take over ownership of an existing session by claiming the lease for
/// <paramref name="claimingPod"/>. Takeover succeeds only when the session exists and
/// its current lease has already expired.
/// </summary>
/// <param name="playSessionId">The play session identifier.</param>
/// <param name="claimingPod">The name of the pod attempting to claim ownership.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>
/// <c>true</c> if the takeover succeeded (the claiming pod now holds the lease);
/// <c>false</c> if the session does not exist, its lease is still valid, or another
/// concurrent caller already claimed it.
/// </returns>
Task<bool> TryTakeoverAsync(string playSessionId, string claimingPod, CancellationToken cancellationToken = default);
/// <summary>
/// Persists a new or updated transcoding session.
/// </summary>
/// <param name="session">The session to store.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A <see cref="Task"/> representing the asynchronous operation.</returns>
Task SetAsync(TranscodeSession session, CancellationToken cancellationToken = default);
/// <summary>
/// Renews the lease for an existing session, extending its
/// <see cref="TranscodeSession.LeaseExpiresUtc"/> by the store's configured lease duration.
/// The renewal is rejected when <paramref name="ownerPod"/> no longer owns the lease, so a
/// renewal in flight while another pod wins <see cref="TryTakeoverAsync"/> cannot revert the takeover.
/// </summary>
/// <param name="playSessionId">The play session identifier.</param>
/// <param name="ownerPod">The name of the pod that believes it owns the lease.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>
/// <c>true</c> if the lease was renewed; <c>false</c> if the session no longer exists or is
/// owned by another pod.
/// </returns>
Task<bool> RenewLeaseAsync(string playSessionId, string ownerPod, CancellationToken cancellationToken = default);
/// <summary>
/// Removes a transcoding session from the store.
/// </summary>
/// <param name="playSessionId">The play session identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A <see cref="Task"/> representing the asynchronous operation.</returns>
Task DeleteAsync(string playSessionId, CancellationToken cancellationToken = default);
/// <summary>
/// Returns all currently active transcoding sessions from the store.
/// </summary>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>
/// An enumerable of <see cref="TranscodeSession"/> objects representing all active sessions.
/// Returns an empty enumerable if no sessions are active or if the store cannot be reached.
/// </returns>
Task<IEnumerable<TranscodeSession>> GetActiveSessionsAsync(CancellationToken cancellationToken = default);
}

Some files were not shown because too many files have changed in this diff Show More