UserDataManager cache causes lost updates across replicas #13

New Issue

2026-09-13T13:33:15+10:00

unkin-agent commented

2026-09-13 13:33:15 +10:00

Context: multi-replica stateless deployment (two 2-replica apps, cheeztv and fafflix, each one PostgreSQL plus valkey, no sticky sessions). Not a 12.0 regression — live in the deployed build. Tracking: #12.

Problem

Emby.Server.Implementations/Library/UserDataManager.cs:30 holds an LRU cache of user data. Reads at :189 return the cached entry with no freshness comparison against the database; writes go through :82.

With two replicas and no sticky sessions, pod B serves a stale resume position out of its own cache and then writes it back over pod A's newer value. Resume positions, played/unplayed state, favourites and ratings are silently lost. This is user-visible data loss, happening in production now.

Proposal

Cache-aside with a real invalidation signal: either a RowVersion/concurrency token compared on read, or valkey pub/sub fan-out on SaveUserData.

Alternatively drop the cache entirely, as was done for DeviceManager, where the measured cost was one indexed single-row SELECT per request. The same may well be acceptable here, but this path is hotter (a SELECT per playback progress tick), so the trade-off deserves measuring before the cache is removed.

LibraryManager's item cache (#19) needs the same invalidation mechanism — likely one piece of work with this.

Acceptance

Two independently constructed UserDataManager instances against real PostgreSQL: A writes a new resume position; B, which has already cached the old value, reads the new one; a subsequent write from B does not revert A's value.
Test uses PostgreSqlTestServer in tests/Jellyfin.Server.Tests and runs in the postgres-migration-chain Woodpecker step (JELLYFIN_TEST_POSTGRES). In-memory providers have hidden different semantics and are how three bugs reached production, so an in-memory-only test does not close this.
Whichever trade-off is chosen, the PR records a measurement of the hot-path cost.

Context: multi-replica stateless deployment (two 2-replica apps, `cheeztv` and `fafflix`, each one PostgreSQL plus valkey, no sticky sessions). Not a 12.0 regression — live in the deployed build. Tracking: #12. ## Problem `Emby.Server.Implementations/Library/UserDataManager.cs:30` holds an LRU cache of user data. Reads at `:189` return the cached entry with no freshness comparison against the database; writes go through `:82`. With two replicas and no sticky sessions, pod B serves a stale resume position out of its own cache and then writes it back over pod A's newer value. Resume positions, played/unplayed state, favourites and ratings are silently lost. This is user-visible data loss, happening in production now. ## Proposal Cache-aside with a real invalidation signal: either a `RowVersion`/concurrency token compared on read, or valkey pub/sub fan-out on `SaveUserData`. Alternatively drop the cache entirely, as was done for `DeviceManager`, where the measured cost was one indexed single-row SELECT per request. The same may well be acceptable here, but this path is hotter (a SELECT per playback progress tick), so the trade-off deserves measuring before the cache is removed. `LibraryManager`'s item cache (#19) needs the same invalidation mechanism — likely one piece of work with this. ## Acceptance - Two independently constructed `UserDataManager` instances against real PostgreSQL: A writes a new resume position; B, which has already cached the old value, reads the new one; a subsequent write from B does not revert A's value. - Test uses `PostgreSqlTestServer` in `tests/Jellyfin.Server.Tests` and runs in the `postgres-migration-chain` Woodpecker step (`JELLYFIN_TEST_POSTGRES`). In-memory providers have hidden different semantics and are how three bugs reached production, so an in-memory-only test does not close this. - Whichever trade-off is chosen, the PR records a measurement of the hot-path cost.

unkin-agent referenced this issue

2026-09-13 13:33:42 +10:00

LibraryManager item cache serves stale metadata across replicas #19

unkin-agent referenced this issue

2026-09-13 13:33:42 +10:00

HA: in-process single-instance assumptions under multi-replica deployment (tracking) #12

Sign in to join this conversation.