remotes: add least-connections mirror strategy (round-robin remains default) #122
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: unkin/artifactapi#122
Reference in New Issue
Block a user
Delete Branch "benvin/mirrorlist-least-conn"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
The mirrorlist (PR #121) always load-balances round-robin. Round-robin is oblivious to how busy each mirror is, so a slow or saturated mirror keeps getting its fair share of new requests. This adds an opt-in least-connections strategy that favors the mirror currently handling the fewest in-flight requests, steering new work toward idle mirrors. Round-robin stays the default, so existing remotes are unchanged.
How
models.Remotegainsmirror_strategy(round_robindefault/empty, orleast_conn).ValidateMirrorStrategychecks the enum and requires a non-empty mirrorlist forleast_conn. Empty behaves asround_robinfor back-compat.mirror_strategy TEXT NOT NULL DEFAULT 'round_robin'column (CREATE TABLE +ADD COLUMN IF NOT EXISTS), wired through remoteCols/scanRemote/CreateRemote/UpdateRemote; empty normalized toround_robinon write.least_connremote the attempt order starts with the least-loaded pool URL (ties broken by the existing round-robin rotation). Round-robin path and failover order are unchanged; single-URL pools are a no-op.least_conndistribution test plus a realdnfinstall through aleast_connremote.Back-compat: unset/empty
mirror_strategyisround_robin, so all existing remotes keep their current behavior.Add a per-remote mirror_strategy selector for the mirrorlist, supporting round_robin (default, unchanged) and least_conn. - models.Remote gains MirrorStrategy string + MirrorStrategy{RoundRobin,LeastConn} constants and ValidateMirrorStrategy (enum check; least_conn requires a non-empty mirrorlist). Empty behaves as round_robin for back-compat. - DB: additive mirror_strategy TEXT NOT NULL DEFAULT 'round_robin' column (CREATE TABLE + ADD COLUMN IF NOT EXISTS), wired through remoteCols/scanRemote/ CreateRemote/UpdateRemote; empty normalized to round_robin on write. - Engine: least_conn starts each attempt with the pool URL holding the fewest in-flight requests via a per-remote/per-URL atomic gauge (incremented around each upstream call in head/fetch/checkUpstream), ties broken by the existing round-robin rotation. Round-robin path and failover order unchanged; single-URL pools are a no-op. - Tests: unit tests for least-loaded selection, round-robin default, gauge inc/dec, single-URL no-op, and strategy validation; DB round-trip covers the new column; docker e2e adds a least_conn distribution test and a real dnf install through a least_conn remote.