unkin-agent
ac8be0a105
OS package remotes (rpm/deb/apk) fetch many small files and benefit from
spreading upstream load across mirrors and surviving a mirror outage. A
remote may now set a `mirrorlist` of additional upstream base URLs; the
effective upstream pool is [base_url] + mirrorlist, which the shared proxy
engine load-balances round-robin and, on a network error/timeout/5xx,
fails over to the next mirror before returning an error. Because selection
happens in the engine, it works for every provider that reaches upstream.
Backward compatible: `base_url` stays a plain string that providers read
unchanged, and a remote with no mirrorlist behaves exactly as today
(single attempt, same error path).
- add models.Remote.Mirrorlist ([]string, json "mirrorlist,omitempty") and
UpstreamPool() = [base_url] + mirrorlist; ValidateMirrorlist enforces
remote repo_type + package_type in {rpm, deb, alpine} and http/https URLs
- v2 create/update: reject a mirrorlist on any other repo (400); base_url
remains required for remotes
- persist mirrorlist in a new additive `mirrorlist TEXT[]` column
(remoteCols/scanRemote/CreateRemote/UpdateRemote); base_url column
unchanged
- engine: per-remote round-robin cursor over the pool; wrap the
fetch/head/revalidate upstream calls in a failover loop that narrows the
remote to one selected mirror per attempt; only network errors and 5xx
fail over (404/403/... return as-is); circuit breaker stays keyed per
remote and trips only after all mirrors fail
- tests: model JSON round-trip + validation gating, engine
round-robin/failover/no-mirrorlist-unchanged, DB mirrorlist round-trip,
and a docker acceptance suite (round-robin across two mock upstreams,
failover past a dead primary, no-mirrorlist regression, and a real dnf
makecache+install through a two-mirror rpm remote whose base_url is dead)
Least-connections and a per-remote strategy selector are a follow-up PR.
35 lines
1.3 KiB
YAML
35 lines
1.3 KiB
YAML
# Overlay for the dockerised end-to-end suite (scripts/docker-e2e.sh).
|
|
# Adds a static mock upstream that the artifactapi container proxies, so the
|
|
# caching tests are hermetic and need no internet access.
|
|
services:
|
|
mockupstream:
|
|
image: nginx:alpine
|
|
volumes:
|
|
- ./e2e-docker/fixtures:/usr/share/nginx/html:ro,z
|
|
# No host port needed: only the artifactapi container talks to it, and the
|
|
# tests compare served bytes against the on-disk fixtures.
|
|
|
|
# Two constant-body upstreams for the multi-base_url suite: each returns a
|
|
# distinct, upstream-identifying body for any path, so round-robin
|
|
# distribution across a two-mirror remote is directly observable.
|
|
mockupstreama:
|
|
image: nginx:alpine
|
|
volumes:
|
|
- ./e2e-docker/mirror-conf/a.conf:/etc/nginx/conf.d/default.conf:ro,z
|
|
mockupstreamb:
|
|
image: nginx:alpine
|
|
volumes:
|
|
- ./e2e-docker/mirror-conf/b.conf:/etc/nginx/conf.d/default.conf:ro,z
|
|
|
|
artifactapi:
|
|
# The host port is set via ARTIFACTAPI_PORT (see scripts/docker-e2e.sh),
|
|
# defaulting to 8000; the e2e run uses 8001 to avoid colliding with a
|
|
# locally-running instance.
|
|
depends_on:
|
|
mockupstream:
|
|
condition: service_started
|
|
mockupstreama:
|
|
condition: service_started
|
|
mockupstreamb:
|
|
condition: service_started
|