unkin-agent
ac8be0a105
OS package remotes (rpm/deb/apk) fetch many small files and benefit from
spreading upstream load across mirrors and surviving a mirror outage. A
remote may now set a `mirrorlist` of additional upstream base URLs; the
effective upstream pool is [base_url] + mirrorlist, which the shared proxy
engine load-balances round-robin and, on a network error/timeout/5xx,
fails over to the next mirror before returning an error. Because selection
happens in the engine, it works for every provider that reaches upstream.
Backward compatible: `base_url` stays a plain string that providers read
unchanged, and a remote with no mirrorlist behaves exactly as today
(single attempt, same error path).
- add models.Remote.Mirrorlist ([]string, json "mirrorlist,omitempty") and
UpstreamPool() = [base_url] + mirrorlist; ValidateMirrorlist enforces
remote repo_type + package_type in {rpm, deb, alpine} and http/https URLs
- v2 create/update: reject a mirrorlist on any other repo (400); base_url
remains required for remotes
- persist mirrorlist in a new additive `mirrorlist TEXT[]` column
(remoteCols/scanRemote/CreateRemote/UpdateRemote); base_url column
unchanged
- engine: per-remote round-robin cursor over the pool; wrap the
fetch/head/revalidate upstream calls in a failover loop that narrows the
remote to one selected mirror per attempt; only network errors and 5xx
fail over (404/403/... return as-is); circuit breaker stays keyed per
remote and trips only after all mirrors fail
- tests: model JSON round-trip + validation gating, engine
round-robin/failover/no-mirrorlist-unchanged, DB mirrorlist round-trip,
and a docker acceptance suite (round-robin across two mock upstreams,
failover past a dead primary, no-mirrorlist regression, and a real dnf
makecache+install through a two-mirror rpm remote whose base_url is dead)
Least-connections and a per-remote strategy selector are a follow-up PR.
51 lines
2.1 KiB
Bash
Executable File
51 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Build the artifactapi container, bring up the full stack (postgres, redis,
|
|
# minio, artifactapi) plus a static mock upstream, and run the dockerised e2e
|
|
# suite against the running product over HTTP. Tears everything down on exit.
|
|
set -euo pipefail
|
|
|
|
cd "$(dirname "$0")/.."
|
|
|
|
# Publish artifactapi on 8001 to avoid colliding with a local instance on 8000.
|
|
export ARTIFACTAPI_PORT="${ARTIFACTAPI_PORT:-8001}"
|
|
COMPOSE=(docker compose -f docker-compose.yml -f docker-compose.e2e.yml)
|
|
API_URL="${ARTIFACTAPI_URL:-http://localhost:${ARTIFACTAPI_PORT}}"
|
|
|
|
cleanup() {
|
|
echo "==> tearing down stack"
|
|
"${COMPOSE[@]}" down -v --remove-orphans >/dev/null 2>&1 || true
|
|
}
|
|
trap cleanup EXIT
|
|
|
|
echo "==> building and starting stack (postgres, redis, minio, mockupstream(s), artifactapi)"
|
|
"${COMPOSE[@]}" up -d --build postgres redis minio mockupstream mockupstreama mockupstreamb artifactapi
|
|
|
|
echo "==> waiting for artifactapi health at ${API_URL}"
|
|
for i in $(seq 1 60); do
|
|
if curl -fsS "${API_URL}/health" >/dev/null 2>&1; then
|
|
echo " healthy after ${i}s"
|
|
break
|
|
fi
|
|
if [ "$i" -eq 60 ]; then
|
|
echo "!!! artifactapi did not become healthy in time; recent logs:"
|
|
"${COMPOSE[@]}" logs --tail=50 artifactapi
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
# Resolve the compose network the artifactapi container is attached to, so the
|
|
# real-package-manager test can launch a stock distro container on the same
|
|
# network and reach artifactapi by service name.
|
|
API_CID="$("${COMPOSE[@]}" ps -q artifactapi)"
|
|
COMPOSE_NETWORK="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{end}}' "${API_CID}" 2>/dev/null || true)"
|
|
|
|
echo "==> running dockerised e2e suite (compose network: ${COMPOSE_NETWORK:-unknown})"
|
|
ARTIFACTAPI_URL="${API_URL}" \
|
|
MOCK_UPSTREAM_INTERNAL="${MOCK_UPSTREAM_INTERNAL:-http://mockupstream}" \
|
|
MOCK_UPSTREAM_A_INTERNAL="${MOCK_UPSTREAM_A_INTERNAL:-http://mockupstreama}" \
|
|
MOCK_UPSTREAM_B_INTERNAL="${MOCK_UPSTREAM_B_INTERNAL:-http://mockupstreamb}" \
|
|
ARTIFACTAPI_INTERNAL="${ARTIFACTAPI_INTERNAL:-http://artifactapi:8000}" \
|
|
COMPOSE_NETWORK="${COMPOSE_NETWORK}" \
|
|
go test -tags=dockere2e -count=1 -timeout=10m -v ./e2e-docker/...
|