diff --git a/Dockerfile.runtime b/Dockerfile.runtime index 87dec4942..3147cb2ae 100644 --- a/Dockerfile.runtime +++ b/Dockerfile.runtime @@ -1,7 +1,6 @@ # syntax=docker/dockerfile:1 # Runtime-only image — the .NET publish step runs on the CI host (runner), -# not inside this Dockerfile. This avoids DinD overlay-on-overlay I/O throttling -# which makes `dotnet publish` inside Docker-in-Docker prohibitively slow on k3s. +# not inside this Dockerfile. # ── Web client stage ────────────────────────────────────────────────────────── # Install jellyfin-web via the official Jellyfin apt repo. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 000000000..454e5d048 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,202 @@ +> **Last updated: 2026-03-04** + +# Jellyfin Server Architecture + +High-level overview of the Jellyfin server structure, layer responsibilities, and key subsystems. + +## Runtime + +| Component | Value | +|---|---| +| Framework | .NET 10 / ASP.NET Core 10 | +| Target | `net10.0` | +| Entry point | `Jellyfin.Server` | +| Version | `10.12.0` (see `SharedVersion.cs`) | + +--- + +## Layer Diagram + +``` +┌───────────────────────────────────────────────────────────┐ +│ HTTP Clients │ +│ (Jellyfin Web, mobile apps, 3rd-party) │ +└────────────────────────┬──────────────────────────────────┘ + │ REST / WebSocket +┌────────────────────────▼──────────────────────────────────┐ +│ Jellyfin.Api │ +│ ASP.NET Core controllers, middleware, auth, Swashbuckle │ +└────────────────────────┬──────────────────────────────────┘ + │ Interfaces (ILibraryManager, etc.) +┌────────────────────────▼──────────────────────────────────┐ +│ MediaBrowser.Controller │ +│ Core domain interfaces — no implementation here │ +└────────────────────────┬──────────────────────────────────┘ + │ Implementations +┌────────────────────────▼──────────────────────────────────┐ +│ Emby.Server.Implementations / Jellyfin.Server.Impl │ +│ Library manager, item repos, scheduled tasks, HTTP server│ +└────────┬───────────────────────────────┬──────────────────┘ + │ │ +┌────────▼────────┐ ┌────────▼────────┐ +│ Jellyfin.Data │ │ MediaBrowser │ +│ EF Core DbCtx │ │ MediaEncoding │ +│ SQLite via │ │ FFmpeg, HLS, │ +│ Microsoft.Data │ │ Trickplay │ +│ .Sqlite │ └─────────────────┘ +└─────────────────┘ + │ +┌────────▼─────────────────────────────────────────────────┐ +│ MediaBrowser.Model │ +│ Pure DTOs, enums, no logic (shared by all layers) │ +└──────────────────────────────────────────────────────────┘ +``` + +--- + +## Project Responsibilities + +### `Jellyfin.Server` + +Entry point. Handles: +- CLI argument parsing (`CommandLineParser`) +- Serilog configuration (console, file, Graylog sinks) +- DI container wiring (`ApplicationHost`) +- ASP.NET Core host startup + +### `Jellyfin.Api` + +All HTTP surface. Handles: +- ASP.NET Core controllers (`Controllers/`) +- Authentication middleware (`Auth/`) +- Swashbuckle/OpenAPI configuration +- Request/response formatting (camelCase + PascalCase JSON) +- WebSocket listeners (`WebSocketListeners/`) + +Controllers inherit from `BaseJellyfinApiController` which sets default route, produces JSON, and provides typed `Ok()` helpers. + +### `MediaBrowser.Controller` + +Core domain interfaces. Key examples: +- `ILibraryManager` — media library operations +- `IMediaEncoder` — FFmpeg wrapper +- `IProviderManager` — metadata provider coordination +- `IUserManager` — user management +- `IPlaybackManager` — playback session tracking + +**No implementations live here.** This keeps the domain decoupled from infrastructure. + +### `Emby.Server.Implementations` + +Primary implementation assembly. Contains: +- `ApplicationHost.cs` — DI wiring and startup +- `Data/` — SQLite queries and EF Core repositories +- `Library/` — `LibraryManager`, `LibraryMonitor` +- `Images/` — image processing pipeline (SkiaSharp) +- `HttpServer/` — HTTP server wiring + +### `Jellyfin.Server.Implementations` + +Secondary implementation assembly split from `Emby.Server.Implementations`. Contains newer implementations using EF Core patterns. + +### `Jellyfin.Data` + +EF Core data models and `DbContext`. Migrations managed here. + +### `MediaBrowser.Model` + +Pure data-transfer objects (DTOs) and enums. No logic. Consumed by all layers and by external clients. Changes here are API-breaking. + +### `MediaBrowser.Providers` + +Online metadata providers: +- TMDB (movies, TV) +- MusicBrainz (audio) +- OMDB +- TV Maze, TheTVDB + +Uses `IMetadataProvider` interface from `MediaBrowser.Controller`. + +### `MediaBrowser.MediaEncoding` + +FFmpeg process management, HLS streaming, keyframe extraction, subtitle transcoding, trickplay image generation. + +### `Emby.Naming` + +Media file path parsing — resolves series/season/episode structure, detects extras, parses video codecs from filenames. + +### `MediaBrowser.LocalMetadata` / `MediaBrowser.XbmcMetadata` + +Local NFO/XML metadata providers (Kodi-compatible `.nfo` sidecar files). + +### `src/Jellyfin.CodeAnalysis` + +Custom Roslyn analyzer. Runs only in Debug builds. Enforces project-specific rules. + +--- + +## Key Subsystems + +### Authentication + +- Session-based API keys (stored in SQLite) +- Quick Connect (pairing flow) +- Auth middleware in `Jellyfin.Api/Auth/` +- Policies defined in `Jellyfin.Api/Constants/Policies.cs` + +### Library Scanning + +1. `LibraryMonitor` watches filesystem for changes +2. `LibraryManager` resolves paths → `BaseItem` subclasses +3. `Emby.Naming` parses filenames → metadata hints +4. `IProviderManager` fetches remote metadata and saves locally +5. Results persisted to SQLite via EF Core + +### Transcoding + +1. Client requests a stream via `MediaInfoController` or `DynamicHlsController` +2. `MediaInfoHelper` determines if transcoding is needed (codec matrix) +3. `MediaEncoder` spawns an FFmpeg subprocess with computed arguments +4. HLS segments or direct stream served via `AudioController` / `VideosController` + +### Metrics + +prometheus-net serves metrics at `/metrics`. Key meters: +- `prometheus-net.AspNetCore` — HTTP request duration/count +- `prometheus-net.DotNetRuntime` — GC, thread pool, JIT metrics +- Custom counters can be added via `Metrics.CreateCounter(...)` in any service + +### Logging + +Serilog pipeline: +- Console sink (structured) +- File sink (rolling, default `%APPDATA%/jellyfin/logs/`) +- Graylog GELF sink (optional, configured via `logging.json`) + +--- + +## Database + +SQLite database at `{DataDir}/data/jellyfin.db`. Accessed via: +- EF Core (`Jellyfin.Data.JellyfinDbContext`) for new data access +- `Microsoft.Data.Sqlite` direct queries for legacy paths + +**All EF Core operations must use async methods** (`ToListAsync`, `FirstOrDefaultAsync`, etc.). + +--- + +## Test Layout + +``` +tests/ + Jellyfin.Api.Tests/ Controller + middleware unit tests + Jellyfin.Common.Tests/ MediaBrowser.Common utilities + Jellyfin.Controller.Tests/ Interface contracts and helpers + Jellyfin.MediaEncoding.Tests/ FFmpeg argument building + Jellyfin.Naming.Tests/ File path parsing + Jellyfin.Providers.Tests/ Provider logic + Jellyfin.Server.Integration.Tests/ Full-stack HTTP tests + OpenAPI spec gen + Jellyfin.Server.Tests/ Server startup and DI tests +``` + +Test stack: xUnit + AutoFixture + Moq + FsCheck. See `.github/instructions/testing.instructions.md`. diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 000000000..7c360ff56 --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,164 @@ +> **Last updated: 2026-03-04** + +# Contributing to Jellyfin Server + +This guide covers everything you need to develop, build, test, and submit changes to the Jellyfin server. + +## Prerequisites + +| Tool | Version | Notes | +|---|---|---| +| .NET SDK | 10.0.x | See `global.json` — `rollForward: latestMinor` | +| Git | any recent | `git clone` with submodules not required | +| FFmpeg | 7.x | Required for transcoding tests; install via devcontainer or manually | +| Docker | optional | For devcontainer workflow | + +### macOS (Homebrew) + +```bash +brew install dotnet +``` + +### Linux (Debian/Ubuntu) + +```bash +wget https://dot.net/v1/dotnet-install.sh && bash dotnet-install.sh --channel 10.0 +``` + +### Windows + +Download the [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10) installer. + +### DevContainer (recommended for new contributors) + +Open the repo in VS Code and accept the "Reopen in Container" prompt. The devcontainer installs: +- .NET 10 +- FFmpeg +- All recommended VS Code extensions + +--- + +## Build + +```bash +# Build the server entry point +dotnet build Jellyfin.Server/Jellyfin.Server.csproj + +# Build the entire solution (all projects) +dotnet build Jellyfin.sln +``` + +Debug builds activate all code analyzers (StyleCop, BannedApiAnalyzers, IDisposableAnalyzers, MultithreadingAnalyzer). **Expect build failures if your code has missing XML docs or uses banned APIs.** + +--- + +## Run Locally + +```bash +dotnet run --project Jellyfin.Server/Jellyfin.Server.csproj \ + -- --datadir /tmp/jellyfin-data --webdir /tmp/jellyfin-web --nowebclient +``` + +The server starts on `http://localhost:8096` by default. + +--- + +## Test + +```bash +# Run all tests (cross-platform matrix: Linux, macOS, Windows) +dotnet test Jellyfin.sln --configuration Release --verbosity minimal + +# Run a single test project +dotnet test tests/Jellyfin.Api.Tests/Jellyfin.Api.Tests.csproj + +# Run tests matching a name filter +dotnet test Jellyfin.sln --filter "ClassName=MyServiceTests" + +# Run with code coverage +dotnet test Jellyfin.sln \ + --configuration Release \ + --collect:"XPlat Code Coverage" \ + --settings tests/coverletArgs.runsettings +``` + +Coverage output: `merged/Cobertura.xml` (merged by ReportGenerator in CI). + +### Regenerate OpenAPI Spec + +After adding or changing any API endpoint: + +```bash +dotnet test tests/Jellyfin.Server.Integration.Tests/Jellyfin.Server.Integration.Tests.csproj \ + -c Release \ + --filter "Jellyfin.Server.Integration.Tests.OpenApiSpecTests" +``` + +Commit the updated `openapi.json` — the CI diff job will flag unintentional breaking changes. + +--- + +## Code Style + +All style rules are enforced by the compiler in Debug builds. Key rules: + +- **Nullable enabled** — mark nullable types with `?`, never silence with `null!` without a comment +- **Warnings as errors** — fix every warning; do not suppress with `#pragma warning disable` +- **XML docs** — every `public` type and member must have `/// ` +- **No `Task.Result`** — always `await` instead +- **Central NuGet versions** — versions in `Directory.Packages.props` only, never in `.csproj` +- **File-scoped namespaces** — use `namespace Jellyfin.Example;` (not block-scoped) + +See `.github/instructions/csharp.instructions.md` for the full ruleset. + +--- + +## Pull Request Process + +1. Fork the repo and create a feature branch from `master` +2. Make your changes; ensure `dotnet build` and `dotnet test` pass locally +3. Fill out the PR template (`.github/pull_request_template.md`): + - **Changes**: 1–5 sentence summary + - **Issues**: tag with `Fixes #NNN` +4. CI runs automatically: + - `ci-tests.yml` — tests on Linux, macOS, Windows + - `ci-openapi.yml` — OpenAPI diff + - `ci-codeql-analysis.yml` — security scan +5. A maintainer will review and merge + +### Title format + +Use the imperative mood: +- ✅ `Add lyrics endpoint for audio items` +- ✅ `Fix null reference in LibraryController` +- ❌ `Added lyrics endpoint` +- ❌ `Fixed null reference` + +--- + +## Adding a New Package Dependency + +1. Add the version to `Directory.Packages.props`: + ```xml + + ``` +2. Add the reference to the relevant `.csproj` (no `Version=` attribute): + ```xml + + ``` + +**Never** specify both a version in `Directory.Packages.props` AND in the `.csproj` — that causes `NU1008`. + +--- + +## Project Conventions + +See `.github/instructions/` for detailed instructions per concern: + +| Topic | File | +|---|---| +| C# style | `csharp.instructions.md` | +| API controllers | `api.instructions.md` | +| Tests | `testing.instructions.md` | +| CI/CD workflows | `ci-cd.instructions.md` | +| Documentation | `docs.instructions.md` | diff --git a/docs/GITHUB-DISCUSSION-DRAFT.md b/docs/GITHUB-DISCUSSION-DRAFT.md new file mode 100644 index 000000000..4919ccc94 --- /dev/null +++ b/docs/GITHUB-DISCUSSION-DRAFT.md @@ -0,0 +1,83 @@ +# Jellyfin HA transcoding fork: Redis-backed session failover + experimental PostgreSQL provider + +I've been working on a fork of Jellyfin focused on one specific problem: making HLS transcoding survive pod restarts in a multi-replica Kubernetes deployment. + +## What it does + +Right now, Jellyfin assumes transcode state lives in one server process. If that pod dies, active transcodes die with it. This fork adds a small HA layer so transcode ownership can survive a pod restart: + +- A new `ITranscodeSessionStore` abstraction for durable transcode session tracking +- A `RedisTranscodeSessionStore` implementation with lease-based ownership +- Atomic pod takeover using a Redis Lua script when a lease expires +- Lease-aware cleanup so one pod does not delete segments another pod still needs +- A `NullTranscodeSessionStore` fallback, so single-instance deployments behave exactly like upstream with no config changes + +I also added an experimental PostgreSQL provider for shared-database deployments, since SQLite is not a good fit once multiple replicas are involved. + +## What the HA flow looks like + +- Pod A starts an HLS transcode and registers the session in Redis +- Pod A renews the lease while it owns the session +- If Pod A dies, the lease expires +- Pod B receives the next request, atomically claims the expired lease, and resumes from the last completed segment on shared storage +- The client sees a short buffer pause instead of a hard failure + +## How to run it + +There are three practical modes: + +### 1. Single instance + +No config needed. It falls back to the no-op store automatically. + +### 2. Local HA test + +Run two Jellyfin instances against: + +- the same Redis +- the same shared transcode directory + +That is enough to test failover behavior locally. + +### 3. Kubernetes / k3s + +This is the intended deployment model. You need: + +- 2+ Jellyfin replicas +- Redis +- shared RWX storage for transcode output +- shared media storage +- ideally PostgreSQL if you want a proper shared DB setup + +The key config is: + +```text +Jellyfin:TranscodeStore:RedisConnectionString +Jellyfin:TranscodeStore:LeaseDurationSeconds +``` + +Repo and write-up: + +- Source: https://github.com/ZoltyMat/jellyfin-ha +- Full change summary vs upstream: https://github.com/ZoltyMat/jellyfin-ha/blob/main/docs/FORK-DIFF.md +- Write-up with diagrams and k8s manifests: https://blog.zolty.systems/posts/jellyfin-ha-kubernetes + +## What would be required to merge upstream + +I do not expect this to be merged as-is without discussion. If there is interest, I think the realistic path is to split it into small pieces: + +1. Introduce `ITranscodeSessionStore`, `TranscodeSession`, and `NullTranscodeSessionStore` only +2. Add the DI wiring with no behavior change unless configured +3. Add HLS session registration and lease renewal hooks +4. Add lease-aware cleanup in `DeleteTranscodeFileTask` +5. Add takeover logic in the HLS/session path +6. Discuss whether Redis should be the first supported distributed store, or whether the interface should land before any concrete implementation +7. Treat PostgreSQL as a separate discussion entirely + +I think the HA transcode work has a better chance of review if it is separated from the PostgreSQL provider and migration tooling. + +## Why I'm posting it + +I'm not trying to maintain a permanent hard fork. I built this to see whether Jellyfin could be made to behave well in a replicated environment without rewriting major subsystems. The answer seems to be yes, but it needs maintainers to decide whether this kind of deployment is something upstream wants to support. + +If there's interest, I'm happy to break the work into smaller PRs, clean up anything that does not match project direction, and rework the design around maintainer feedback. \ No newline at end of file