QuickConnect fails intermittently across replicas #18

New Issue

2026-09-13T13:33:16+10:00

unkin-agent commented

2026-09-13 13:33:16 +10:00

Context: multi-replica stateless deployment (two 2-replica apps, cheeztv and fafflix, each one PostgreSQL plus valkey, no sticky sessions). Not a 12.0 regression — live in the deployed build. Tracking: #12.

Problem

Emby.Server.Implementations/QuickConnect/QuickConnectManager.cs:33-34 holds _currentRequests and _authorizedSecrets in process, with zero persistence.

The three legs of a quick-connect flow — initiate, authorize, exchange — can each land on a different pod, so the later legs throw ResourceNotFoundException against a pod that never saw the request. The failure is intermittent, which makes it read as a client bug rather than a server one.

Proposal

Move both dictionaries to shared storage, preserving the existing TTL semantics (request expiry, single-use secret consumption):

valkey gives TTL and atomic single-use consumption for free;
the database avoids a second dependency but needs an expiry sweep and careful single-use semantics.

Self-contained — no dependency on the other items in #12, so it can be picked up at any time.

Acceptance

Three independently constructed instances against real PostgreSQL (and valkey, if chosen): initiate on A, authorize on B, exchange on C — the flow succeeds.
An expired request is rejected, and an authorized secret cannot be exchanged twice even when the two attempts hit different instances.
Tests use PostgreSqlTestServer in tests/Jellyfin.Server.Tests and run in the postgres-migration-chain Woodpecker step (JELLYFIN_TEST_POSTGRES). In-memory providers have hidden different semantics and are how three bugs reached production.

Context: multi-replica stateless deployment (two 2-replica apps, `cheeztv` and `fafflix`, each one PostgreSQL plus valkey, no sticky sessions). Not a 12.0 regression — live in the deployed build. Tracking: #12. ## Problem `Emby.Server.Implementations/QuickConnect/QuickConnectManager.cs:33-34` holds `_currentRequests` and `_authorizedSecrets` in process, with zero persistence. The three legs of a quick-connect flow — initiate, authorize, exchange — can each land on a different pod, so the later legs throw `ResourceNotFoundException` against a pod that never saw the request. The failure is intermittent, which makes it read as a client bug rather than a server one. ## Proposal Move both dictionaries to shared storage, preserving the existing TTL semantics (request expiry, single-use secret consumption): - valkey gives TTL and atomic single-use consumption for free; - the database avoids a second dependency but needs an expiry sweep and careful single-use semantics. Self-contained — no dependency on the other items in #12, so it can be picked up at any time. ## Acceptance - Three independently constructed instances against real PostgreSQL (and valkey, if chosen): initiate on A, authorize on B, exchange on C — the flow succeeds. - An expired request is rejected, and an authorized secret cannot be exchanged twice even when the two attempts hit different instances. - Tests use `PostgreSqlTestServer` in `tests/Jellyfin.Server.Tests` and run in the `postgres-migration-chain` Woodpecker step (`JELLYFIN_TEST_POSTGRES`). In-memory providers have hidden different semantics and are how three bugs reached production.

unkin-agent referenced this issue

2026-09-13 13:33:42 +10:00

HA: in-process single-instance assumptions under multi-replica deployment (tracking) #12

~~unkin-agent referenced this issue 2026-09-24 23:05:08 +10:00~~

fix(ha): share quick connect state between instances #34

unkin-agent referenced this issue

2026-09-26 14:51:54 +10:00

fix(ha): share quick connect state between instances #34

unkin-agent referenced this issue

2026-09-26 18:34:05 +10:00

QuickConnect authorize writes the token and the request flag in two unguarded round trips #35

unkin-agent referenced this issue