fix: make transcode leases ownership-checked and cleanup-aware

Cleanup never matched a live session because the controller registered empty
manifest and segment paths, renewal was a read-modify-write that could revert a
takeover, and the takeover script compared an ISO date to a number, so it errored.

- populate the session record's manifest and segment paths from the playlist path
- renew the lease via a Lua compare-and-set on the owning pod
- store the lease expiry as unix milliseconds so the scripts can compare it
- retain the session record past its lease so an orphan can still be taken over
- test the Redis store against a real Redis, including the renew-vs-takeover race
- drop the live stream record nothing ever read back
This commit is contained in:
2026-09-12 10:19:29 +10:00
parent d825f8ac81
commit baa16b6586
22 changed files with 644 additions and 1221 deletions
@@ -47,11 +47,17 @@ public interface ITranscodeSessionStore
/// <summary>
/// Renews the lease for an existing session, extending its
/// <see cref="TranscodeSession.LeaseExpiresUtc"/> by the store's configured lease duration.
/// The renewal is rejected when <paramref name="ownerPod"/> no longer owns the lease, so a
/// renewal in flight while another pod wins <see cref="TryTakeoverAsync"/> cannot revert the takeover.
/// </summary>
/// <param name="playSessionId">The play session identifier.</param>
/// <param name="ownerPod">The name of the pod that believes it owns the lease.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A <see cref="Task"/> representing the asynchronous operation.</returns>
Task RenewLeaseAsync(string playSessionId, CancellationToken cancellationToken = default);
/// <returns>
/// <c>true</c> if the lease was renewed; <c>false</c> if the session no longer exists or is
/// owned by another pod.
/// </returns>
Task<bool> RenewLeaseAsync(string playSessionId, string ownerPod, CancellationToken cancellationToken = default);
/// <summary>
/// Removes a transcoding session from the store.
@@ -70,35 +76,4 @@ public interface ITranscodeSessionStore
/// Returns an empty enumerable if no sessions are active or if the store cannot be reached.
/// </returns>
Task<IEnumerable<TranscodeSession>> GetActiveSessionsAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Persists a live stream session record so that takeover pods can identify and close
/// streams that were opened on a pod that has since crashed or been evicted.
/// </summary>
/// <param name="session">The live stream session to store.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A <see cref="Task"/> representing the asynchronous operation.</returns>
Task SetLiveStreamAsync(LiveStreamSession session, CancellationToken cancellationToken = default);
/// <summary>
/// Attempts to retrieve a live stream session by its live stream identifier and the
/// session or play-session identifier that owns it.
/// </summary>
/// <param name="liveStreamId">The live stream identifier.</param>
/// <param name="sessionIdOrPlaySessionId">The session identifier or play-session identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>
/// The <see cref="LiveStreamSession"/> if it exists; otherwise <c>null</c>.
/// </returns>
Task<LiveStreamSession?> TryGetLiveStreamAsync(string liveStreamId, string sessionIdOrPlaySessionId, CancellationToken cancellationToken = default);
/// <summary>
/// Removes the live stream session record for the given live stream and session identifier.
/// This is called when the stream is closed, either by the owning pod or a takeover pod.
/// </summary>
/// <param name="liveStreamId">The live stream identifier.</param>
/// <param name="sessionIdOrPlaySessionId">The session identifier or play-session identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A <see cref="Task"/> representing the asynchronous operation.</returns>
Task DeleteLiveStreamAsync(string liveStreamId, string sessionIdOrPlaySessionId, CancellationToken cancellationToken = default);
}