unkin-agent
68a1f14e17
A remote's base_url may now be a single string OR a list of upstream mirrors. When it is a list the shared proxy engine load-balances across them round-robin and, on an upstream error/timeout/5xx, fails over to the next mirror before returning an error. Because the selection happens in the engine (not per provider), it applies to every remote package type. Backward compatible: a bare-string base_url behaves exactly as before. - add models.StringOrSlice (string-or-array JSON) and custom Remote (Un)MarshalJSON: base_url populates BaseURLs (full list) + BaseURL (active/first); marshals a single mirror back to a bare string - add Remote.BaseURLList / ValidateBaseURLs; validate list is non-empty and every entry is an http/https URL in the v2 create/update handlers - persist the full list in a new base_urls TEXT[] column (additive migration), keeping base_url in sync for old readers; only write base_urls for genuinely multi-mirror remotes - engine: per-remote round-robin cursor + attempt ordering; wrap the fetch/head/revalidate upstream calls in a failover loop that narrows the remote to one selected mirror per attempt; only network errors and 5xx fail over (404/403/... are returned as-is); circuit breaker stays keyed per remote and trips only after all mirrors fail - add unit tests (JSON round-trip, engine round-robin/failover/single-URL, DB multi-URL round-trip) and a docker acceptance suite: round-robin distribution across two mock upstreams, failover past a dead primary, single-base_url regression, and a real dnf makecache+install through a two-mirror rpm remote whose primary is dead Least-connections and a per-remote strategy selector are a follow-up PR.
Dockerised end-to-end suite
Black-box tests that run against a fully containerised artifactapi stack
(built image + Postgres + Redis + MinIO) plus a static mock upstream. Unlike the
in-process e2e/ suite (testcontainers, server run in-process), these only speak
HTTP to the running product, so they exercise the shipped container image.
Run
make docker-e2e # build image, compose up, run suite, compose down
scripts/docker-e2e.sh builds and starts docker-compose.yml +
docker-compose.e2e.yml, waits for /health, then runs
go test -tags=dockere2e ./e2e-docker/... and tears everything down.
The stack publishes artifactapi on host port 8001 (to avoid colliding with a
local instance on 8000). Override with ARTIFACTAPI_URL to point the tests at an
already-running stack.
Coverage
- Repository lifecycle — add / change / delete for remote, local and virtual repos.
- Caching — one immutable artifact per remote package type (generic, docker,
helm, pypi, npm, rpm, alpine, puppet, terraform, goproxy) proxied through the
mock upstream: first fetch
X-Artifact-Source: remote, secondcache, bytes verified against the origin fixture. - Local uploads — generic (upload/download), pypi (wheel + generated
simple/index), rpm (real package + automatic repodata generation). - Virtual repositories — pypi simple-index merge and helm
index.yamlmerge across two members. - Multiple base_urls — a remote whose
base_urlis a list of two upstreams: round-robin distribution across both mirrors (constant-bodymockupstreama/mockupstreamb), failover past a dead primary, single-base_url regression, and a realdnf(stockrockylinux:9container)makecache+installthrough a two-mirror rpm remote with a dead primary — a dead mirror must not break the client.
Fixtures
fixtures/ is served by the mock upstream at its web root. Paths mirror each
provider's upstream URL layout (e.g. v2/... for docker, v1/providers/... for
terraform). The RPM under fixtures/rpmrepo/Packages/ is a real package so the
rpm provider can parse its metadata for repodata generation.