Document the repodata fingerprint invariant (#178)

The cache_repodata fingerprint (count, max id) only detects every change while one writer at a time touches a remote's rows; with interleaved writers a late-committing lower id plus a delete can leave it unchanged. In the deployed server, github_rpm scans run under the per-remote sync lease, which holds only while the lease outlives the scan timeout.

- note the invariant on RPMMetadataFingerprint, scoped to github_rpm remotes
- state it in the Repodata cache section of the github_rpm docs
- add a test asserting syncLeaseDuration > defaultScanTimeout

Reviewed-on: #178
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>

README.md

@@ -23,10 +23,11 @@ API: `http://localhost:8000` | Frontend: `http://localhost:5173`
 Providers classify paths automatically. Users only configure what to proxy and TTLs.
 
 Classification precedence, TTLs and eviction for every remote: [docs/common/caching.md](docs/common/caching.md).
+Write authentication and per-client token usage: [docs/common/auth.md](docs/common/auth.md).
 
 Each type's paths, Terraform examples and local/virtual behaviour are in `docs/<type>/`:
 
-- `alpine`: [remote](docs/alpine/remote.md), [remote-github](docs/alpine/remote-github.md)
+- `alpine`: [remote](docs/alpine/remote.md), [remote-github](docs/alpine/remote-github.md), [virtual](docs/alpine/virtual.md)
 - `ansible`: [remote](docs/ansible/remote.md), [local](docs/ansible/local.md), [virtual](docs/ansible/virtual.md)
 - `cargo`: [remote](docs/cargo/remote.md)
 - `deb`: [remote](docs/deb/remote.md), [remote-github](docs/deb/remote-github.md), [virtual](docs/deb/virtual.md)
@@ -202,14 +203,15 @@ additive and idempotent for the same reason; a test enforces it.
 | `GITHUB_SYNC_RATE` | `1` | `github_rpm` syncer global GitHub request rate (req/s), shared across all remotes. `1`/s = 3600/hr, under an authenticated token's ~5000/hr; unauthenticated (~60/hr) relies on ETag/304 |
 | `GITHUB_SYNC_BURST` | `5` | Token-bucket burst for the shared limiter |
 | `GITHUB_SYNC_WORKERS` | `3` | Concurrent `github_rpm` scan workers |
-| `GITHUB_SYNC_POLL_INTERVAL` | `60` | Base scheduler tick in seconds; per-remote cadence is its `mutable_ttl`, enforced by the DB lease |
+| `GITHUB_SYNC_POLL_INTERVAL` | `60` | Base scheduler tick in seconds; per-remote cadence is its `sync_interval` (phase-spread), else `mutable_ttl`, enforced by the DB lease |
 | `GITHUB_TOKEN` | | Server-level GitHub PAT (fine-grained or classic), sent as `Authorization: Bearer`. Applies to every GitHub request; per-remote creds override it. See [GitHub authentication](#github-authentication) |
 | `GITHUB_APP_ID` | | GitHub App id (App auth mode; mutually exclusive with `GITHUB_TOKEN`) |
 | `GITHUB_APP_INSTALLATION_ID` | | GitHub App installation id |
 | `GITHUB_APP_PRIVATE_KEY` | | GitHub App private key, inline PEM |
 | `GITHUB_APP_PRIVATE_KEY_PATH` | | GitHub App private key, file path (alternative to inline PEM) |
-| `ARTIFACTAPI_OIDC_ISSUER` | | OIDC issuer URL (e.g. Vault `https://vault.service.consul:8200/v1/identity/oidc`). When set, `POST /api/v2/remotes`, `POST /api/v2/virtuals` and `PUT`/`DELETE` on `/api/v2/{remotes,virtuals}/{name}` require `Authorization: Bearer <jwt>` verified against the issuer's JWKS. Uploads, object/file deletes, probe, reads and `/v2` stay open. Unset = no auth |
+| `ARTIFACTAPI_OIDC_ISSUER` | | OIDC issuer URL (e.g. Vault `https://vault.example.com/v1/identity/oidc`). When set, `POST /api/v2/remotes`, `POST /api/v2/virtuals` and `PUT`/`DELETE` on `/api/v2/{remotes,virtuals}/{name}` require a token verified against the issuer's JWKS. Unset = no auth. See [docs/common/auth.md](docs/common/auth.md) |
 | `ARTIFACTAPI_OIDC_AUDIENCE` | | Comma-separated allowed `aud` values (e.g. the Vault OIDC role's client_id). Required when `ARTIFACTAPI_OIDC_ISSUER` is set |
+| `ARTIFACTAPI_AUTH_UPLOADS` | `false` | Also require a token for every local repo write: file uploads/deletes, object evictions, `/api/v1/local` publishes and `/v2` push. Requires `ARTIFACTAPI_OIDC_ISSUER` |
 
 ## Development
 

docs/alpine/remote-github.md

@@ -5,3 +5,16 @@ Shared classification, TTL and eviction rules: [caching](../common/caching.md).
 | Type | Mutable (auto-detected) | Immutable (auto-detected) |
 |---|---|---|
 | `github_alpine` | `APKINDEX.tar.gz` (synthesized) | `.apk` (redirected) |
+
+## Release sync
+
+A background syncer re-scans the GitHub releases list per remote, sharing one
+lease per remote across replicas through `github_alpine_sync_state`.
+
+- **`sync_interval`** (seconds) re-checks once per interval at a stable
+  per-remote offset, so remotes on the same interval spread out instead of
+  syncing together; its first scheduled sync waits for that slot (a cold-cache
+  request still primes at once). Unset, it re-checks no more often than its
+  `mutable_ttl`.
+- A failed scan keeps serving the last good index and retries with exponential
+  backoff.

docs/alpine/virtual.md

@@ -0,0 +1,30 @@
+# `alpine` virtual
+
+An alpine virtual serves one apk repository over local and remote alpine
+members at `/api/v1/virtual/<name>/`.
+
+- `<dir>/APKINDEX.tar.gz` merges each member's index for the same `<dir>`
+  (e.g. `x86_64/`); on a duplicate (P, V, A) the earlier member wins. A member
+  that 404s for that directory is skipped.
+- apk builds download paths itself (`<dir>/<P>-<V>.apk`), so the virtual 302s
+  each download to the member whose index listed it.
+- The merged index is unsigned, like local and `github_alpine` indexes: clients
+  need `--allow-untrusted` (packages keep their own signatures).
+- Any other member error fails the request with 502. Merges are cached for 60s
+  per directory, up to 32 directories per virtual.
+
+```hcl
+resource "artifactapi_virtual" "alpine" {
+  name         = "alpine"
+  package_type = "alpine"
+  members      = [artifactapi_local_alpine.internal.name, artifactapi_remote_github_alpine.tool.name]
+}
+```
+
+```sh
+echo https://artifactapi.example.com/api/v1/virtual/alpine >> /etc/apk/repositories
+apk update --allow-untrusted
+apk add --allow-untrusted mytool
+```
+
+Upload to the local member directly; the virtual is read-only.

docs/ansible/local.md

@@ -2,7 +2,7 @@
 
 A local `ansible` repo is a Galaxy v3 server: `ansible-galaxy collection publish`
 and `ansible-galaxy collection install` work against `/api/v1/local/<repo>`.
-`publish` needs an API key; any value works.
+`publish` needs an API key; any value works unless [uploads are gated](../common/auth.md).
 
 ```bash
 ansible-galaxy collection build

docs/common/auth.md

@@ -0,0 +1,53 @@
+# Write authentication
+
+artifactapi verifies writes against an OIDC issuer's JWKS, such as a Vault
+identity OIDC role. Reads, probes and anonymous pulls never need a token.
+
+| Variable | Gates |
+|---|---|
+| `ARTIFACTAPI_OIDC_ISSUER` + `ARTIFACTAPI_OIDC_AUDIENCE` | creating, updating and deleting remotes and virtuals |
+| `ARTIFACTAPI_AUTH_UPLOADS=true` | also every local repo write: `PUT`/`DELETE /api/v2/remotes/<name>/files/*`, `DELETE /api/v2/{remotes,locals}/<name>/objects/*`, `PUT`/`POST`/`DELETE /api/v1/local/<repo>/*` (npm publish, gem push/yank, ansible-galaxy publish) and Docker Registry V2 push/delete under `/v2` |
+
+Any token from the issuer whose `aud` is in `ARTIFACTAPI_OIDC_AUDIENCE` may
+write to every repo.
+
+## Minting a token
+
+```sh
+vault read -field=token identity/oidc/token/<role> > token
+```
+
+The role's `client_id` must be one of the configured audiences.
+
+## Passing the token
+
+The token is accepted as `Authorization: Bearer <token>`, as `Token <token>`,
+as the password of HTTP Basic credentials with any username, or as a bare
+header value.
+
+```sh
+# curl
+curl -fsSL -H "Authorization: Bearer $(cat token)" --upload-file f.tgz \
+  https://artifactapi.example.com/api/v2/remotes/<repo>/files/f.tgz
+curl -fsSL -u "ci:$(cat token)" --upload-file f.tgz \
+  https://artifactapi.example.com/api/v2/remotes/<repo>/files/f.tgz
+
+# docker / podman / helm (OCI): any username, token as password
+docker login artifactapi.example.com -u ci --password-stdin < token
+helm registry login artifactapi.example.com -u ci --password-stdin < token
+
+# npm (.npmrc)
+//artifactapi.example.com/api/v1/local/<repo>/:_authToken=<token>
+
+# gem
+GEM_HOST_API_KEY="$(cat token)" gem push --host https://artifactapi.example.com/api/v1/local/<repo> mygem-1.0.0.gem
+
+# ansible-galaxy
+ansible-galaxy collection publish c.tar.gz \
+  --server https://artifactapi.example.com/api/v1/local/<repo>/ --api-key "$(cat token)"
+```
+
+With uploads gated, `/v2/` answers anonymous requests with a Bearer challenge
+pointing at `/v2/token`. That endpoint returns the verified Basic password as
+the registry token, or an anonymous token that only reads, so `docker pull`
+keeps working without a login.

docs/common/caching.md

@@ -13,9 +13,11 @@ Every proxied request on a remote is classified as **denied**, **immutable** or
 | `mutable_patterns` | Regexes forcing a path mutable. |
 | `immutable_ttl` | Seconds an immutable path stays fresh. `0` = forever. |
 | `mutable_ttl` | Seconds a mutable path stays fresh. `0` also means never expires. Omitted on create, it defaults to `3600`; omitted on update, the stored value is kept. |
+| `sync_interval` | github_rpm/github_deb/github_alpine only. Seconds between background release re-scans, at a stable per-remote offset. Unset (`0`) re-scans no more often than `mutable_ttl`. |
 | `check_mutable` | When a mutable path's `mutable_ttl` expires, sends a conditional `HEAD` (`If-None-Match`) with the stored upstream ETag. A `304` re-arms the TTL and serves the stored copy; anything else refetches in full. The ETag is kept for `max(10 × mutable_ttl, 7 days)`, re-armed on each `304`, so idle paths age out. It is only stored when `check_mutable` is set and `mutable_ttl` is non-zero, and never for immutable paths. |
 | `stale_on_error` | On an upstream network error, serve the stored copy instead of failing. |
-| `quarantine_enabled` / `quarantine_days` | Stored on the remote but not enforced by the proxy yet. |
+| `serve_stale` | rpm/deb/alpine virtual members only. When a virtual re-merges, use this member's previous index at once and refresh it in the background; the next merge picks up the refresh. The first merge still waits, and so does any merge once the previous index is over an hour old, so a member that keeps failing surfaces its error. Default `false`. |
+| `quarantine_enabled` / `quarantine_days` | Refuses immutable artifacts published less than `quarantine_days` (default `3`) ago. See [Quarantine](#quarantine). |
 | `mirrorlist` / `mirror_strategy` | rpm/deb/alpine only. Upstream pool is `[base_url] + mirrorlist`; `round_robin` (default) rotates the starting mirror, `least_conn` starts at the one with the fewest in-flight requests. Network errors and 5xx fail over to the next mirror; other statuses return as-is. |
 
 ## Precedence
@@ -52,6 +54,24 @@ So `blocklist` beats the allowlist, and `immutable_patterns` beats
 Concurrent misses on one path share a fetch lock. Waiters poll the store for
 up to 5s instead of all going upstream.
 
+## Quarantine
+
+With `quarantine_enabled`, an uncached **immutable** path whose upstream copy
+was published less than `quarantine_days` ago answers `403` and is not cached:
+
+```json
+{"error": "quarantined: published 2026-10-24T09:00:00Z, available from 2026-10-27T09:00:00Z",
+ "published": "2026-10-24T09:00:00Z", "available": "2026-10-27T09:00:00Z"}
+```
+
+- `quarantine_days` must be at least `1` when quarantine is enabled.
+- The publish time is the upstream `Last-Modified` header (capped at now), else the first time artifactapi was asked for the path.
+- If that first-seen lookup fails, the path answers `503` with `Retry-After` instead.
+- Mutable paths (indexes, metadata) are never quarantined.
+- Paths already cached keep serving, including ones cached before quarantine was enabled.
+- A virtual redirects to the member that owns the path, so its `403` reaches the client. A goproxy virtual probing for an unlisted version skips a quarantining member and only redirects to it when no other member serves the version.
+- Indexes still list fresh versions, so a client resolving "latest" may pick one and get the `403`.
+
 ## Eviction
 
 ```sh

docs/deb/remote-github.md

@@ -5,3 +5,16 @@ Shared classification, TTL and eviction rules: [caching](../common/caching.md).
 | Type | Mutable (auto-detected) | Immutable (auto-detected) |
 |---|---|---|
 | `github_deb` | `Packages*`, `Release*` (synthesized) | `.deb` (redirected) |
+
+## Release sync
+
+A background syncer re-scans the GitHub releases list per remote, sharing one
+lease per remote across replicas through `github_deb_sync_state`.
+
+- **`sync_interval`** (seconds) re-checks once per interval at a stable
+  per-remote offset, so remotes on the same interval spread out instead of
+  syncing together; its first scheduled sync waits for that slot (a cold-cache
+  request still primes at once). Unset, it re-checks no more often than its
+  `mutable_ttl`.
+- A failed scan keeps serving the last good index and retries with exponential
+  backoff.

docs/docker/local.md

@@ -10,6 +10,9 @@ docker push        artifactapi.example.com/docker-internal/myapp:latest
 docker pull        artifactapi.example.com/docker-internal/myapp:latest
 ```
 
+With [uploads gated](../common/auth.md), `docker login` with any username and
+a token as the password before pushing; pulls stay anonymous.
+
 The first path segment after `/v2/` is the artifactapi repo name; the remainder
 is the image name. Blobs and manifests are stored through the shared
 content-addressable store (deduplicated by digest, reaped by GC once

docs/gems/local.md

@@ -2,7 +2,7 @@
 
 A local `gems` repo is a RubyGems source: `gem push`, `gem yank`, `gem install`
 and `bundle install` work against `/api/v1/local/<repo>`. `gem push` needs an
-API key; any value works.
+API key; any value works unless [uploads are gated](../common/auth.md).
 
 ```bash
 GEM_HOST_API_KEY=unused gem push --host https://artifactapi.example.com/api/v1/local/<repo> mygem-1.0.0.gem

docs/npm/local.md

@@ -2,7 +2,8 @@
 
 A local `npm` repo is a real npm registry: `npm publish` (scoped and unscoped)
 and `npm install` work against `/api/v1/local/<repo>/`. npm requires an
-`_authToken` for the registry before it will publish; any value works.
+`_authToken` for the registry before it will publish; any value works unless
+[uploads are gated](../common/auth.md).
 
 ```ini
 # .npmrc

docs/rpm/remote-github.md

@@ -32,10 +32,14 @@ derived metadata current off the client request path:
   The first `dnf` request is served from cache. If a request arrives before the
   prime lands, it returns a retryable `503` (with `Retry-After`) rather than
   serving an empty repo or blocking on a multi-minute derive.
-- **Periodic re-check, driven by `mutable_ttl`.** Each remote is re-checked for
+- **Periodic re-check, driven by `sync_interval` or `mutable_ttl`.** With
-  new or changed releases no more often than its `mutable_ttl`. New/changed
+  `sync_interval` (seconds) set, a remote re-checks once per interval at a
-  assets are derived incrementally; assets already cached are never re-fetched,
+  stable per-remote offset, so remotes on the same interval spread out instead
-  and assets that disappear upstream are pruned.
+  of syncing together; its first scheduled sync waits for that slot (a
+  cold-cache request still primes at once). Unset, it re-checks no more often
+  than its `mutable_ttl`. New/changed assets are derived incrementally; assets
+  already cached are never re-fetched, and assets that disappear upstream are
+  pruned.
 - **ETag / 304 conditional requests.** The releases-list `ETag` is stored per
   remote and sent as `If-None-Match`; a `304 Not Modified` means nothing changed
   and the syncer derives nothing. GitHub does not count `304` conditional
@@ -50,8 +54,8 @@ derived metadata current off the client request path:
   periodic scan a replica must atomically claim a per-remote lease
   (`github_rpm_sync_state`: `last_synced_at`, `etag`, `sync_lease_owner`,
   `sync_lease_expires`); only the winner scans. This bounds total GitHub load to
-  ~once per `mutable_ttl` regardless of replica count, and the shared `etag`
+  ~once per `sync_interval` (else `mutable_ttl`) regardless of replica count,
-  lets any replica issue the conditional request.
+  and the shared `etag` lets any replica issue the conditional request.
 
 ```hcl
 # Backend that serves the actual .rpm bytes from github.com.
@@ -68,6 +72,7 @@ resource "artifactapi_remote_github_rpm" "acme-tools" {
   base_url        = "https://api.github.com/repos/acme/tools" # the releases API root
   releases_remote = "github"                                  # backend for downloads
   mutable_ttl     = 3600                                      # release re-scan interval
+  cache_repodata  = true                                      # keep generated repodata in memory
 
   # Optional: restrict which release assets become packages (regex on filename).
   patterns = [".*\\.x86_64\\.rpm$", ".*\\.noarch\\.rpm$"]
@@ -80,3 +85,13 @@ resource "artifactapi_remote_github_rpm" "acme-tools" {
 `dnf` config: `baseurl=https://artifactapi.example/api/v1/remote/acme-tools`.
 The repo is multi-arch (no `$basearch` needed) — `dnf` selects matching packages
 from the synthesized metadata.
+
+### Repodata cache
+
+Repodata is regenerated from the metadata rows on every request by default. Set
+`cache_repodata = true` on the remote to keep the generated `repomd.xml` and its
+`primary`/`filelists`/`other` files in memory per replica. Each request checks a
+cheap row fingerprint (count + newest id) in the database and rebuilds only when
+it moved, so every replica picks up a scan's changes on its next request. The
+fingerprint relies on only one scan writing a remote's rows at a time; in the
+deployed server, the sync lease serialises scans across replicas.

e2e-docker/alpine_virtual_test.go

@@ -0,0 +1,105 @@
+//go:build dockere2e
+
+package e2edocker
+
+import (
+	"net/http"
+	"os"
+	"os/exec"
+	"strings"
+	"testing"
+	"time"
+)
+
+// abuildScript builds a real, abuild-signed $N-$V-r0.apk and writes it to stdout.
+const abuildScript = `set -eu
+apk add -q abuild >&2
+adduser -D b && addgroup b abuild
+su b -c 'set -eu; abuild-keygen -an >&2; mkdir -p ~/p && cd ~/p
+printf "%s\n" "pkgname=$N" "pkgver=$V" pkgrel=0 "pkgdesc=\"e2e\"" "url=\"https://example.com\"" "arch=\"x86_64\"" "license=\"MIT\"" "options=\"!check\"" \
+  "package() { mkdir -p \"\$pkgdir/usr/share/$N\"; echo ok > \"\$pkgdir/usr/share/$N/ok\"; }" > APKBUILD
+abuild -d >&2 || true'
+cat /home/b/packages/*/x86_64/$N-$V-r0.apk`
+
+func buildApk(t *testing.T, name, version string) []byte {
+	t.Helper()
+	cmd := exec.Command("docker", "run", "--rm", "-e", "N="+name, "-e", "V="+version, "alpine:3.20", "sh", "-c", abuildScript)
+	var stderr strings.Builder
+	cmd.Stderr = &stderr
+	out, err := cmd.Output()
+	if err != nil || len(out) == 0 {
+		t.Fatalf("abuild %s-%s: %v\n%s", name, version, err, stderr.String())
+	}
+	return out
+}
+
+// TestVirtualAlpineMerge merges two local alpine repos carrying the same
+// abuild-built package: the first member wins the duplicate and its download,
+// and a real apk installs packages from both members through the virtual.
+func TestVirtualAlpineMerge(t *testing.T) {
+	network := os.Getenv("COMPOSE_NETWORK")
+	internal := os.Getenv("ARTIFACTAPI_INTERNAL")
+	if network == "" || internal == "" {
+		t.Skip("COMPOSE_NETWORK/ARTIFACTAPI_INTERNAL not set; skipping real apk")
+	}
+	if _, err := exec.LookPath("docker"); err != nil {
+		t.Skip("docker not available on the test host")
+	}
+	one, two := buildApk(t, "vapk-one", "1.0"), buildApk(t, "vapk-two", "2.0")
+
+	createRepo(t, `{"name":"vapk-a","package_type":"alpine","repo_type":"local"}`)
+	createRepo(t, `{"name":"vapk-b","package_type":"alpine","repo_type":"local"}`)
+	defer deleteRepo(t, "vapk-a")
+	defer deleteRepo(t, "vapk-b")
+
+	const apkType = "application/vnd.android.package-archive"
+	uploadFile(t, "vapk-a", "x86_64/vapk-one-1.0-r0.apk", one, apkType)
+	uploadFile(t, "vapk-b", "x86_64/vapk-one-1.0-r0.apk", one, apkType)
+	uploadFile(t, "vapk-b", "x86_64/vapk-two-2.0-r0.apk", two, apkType)
+
+	// Local indexes list uploads once their metadata is parsed asynchronously,
+	// and the virtual caches its merge, so merge only once both members list all.
+	for _, m := range []struct{ repo, pkg string }{{"vapk-a", "P:vapk-one\n"}, {"vapk-b", "P:vapk-two\n"}} {
+		var idx string
+		for deadline := time.Now().Add(15 * time.Second); !strings.Contains(idx, m.pkg) && time.Now().Before(deadline); time.Sleep(250 * time.Millisecond) {
+			_, body := doRequest(t, http.MethodGet, api("/api/v1/local/"+m.repo+"/x86_64/APKINDEX.tar.gz"), nil, "")
+			idx = apkIndexText(t, body)
+		}
+	}
+
+	createVirtual(t, `{"name":"vapk","package_type":"alpine","members":["vapk-a","vapk-b"]}`)
+	defer deleteVirtual(t, "vapk")
+
+	resp, body := doRequest(t, http.MethodGet, api("/api/v1/virtual/vapk/x86_64/APKINDEX.tar.gz"), nil, "")
+	if resp.StatusCode != http.StatusOK {
+		t.Fatalf("virtual APKINDEX: status %d: %s", resp.StatusCode, body)
+	}
+	index := apkIndexText(t, body)
+	if strings.Count(index, "P:vapk-one\n") != 1 || !strings.Contains(index, "P:vapk-two\n") {
+		t.Fatalf("merged APKINDEX should hold vapk-one once and vapk-two:\n%s", index)
+	}
+
+	noRedirect := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
+	for pkg, member := range map[string]string{"vapk-one-1.0-r0.apk": "vapk-a", "vapk-two-2.0-r0.apk": "vapk-b"} {
+		resp, err := noRedirect.Get(api("/api/v1/virtual/vapk/x86_64/" + pkg))
+		if err != nil {
+			t.Fatal(err)
+		}
+		resp.Body.Close()
+		if loc := resp.Header.Get("Location"); resp.StatusCode != http.StatusFound || !strings.HasSuffix(loc, "/api/v1/local/"+member+"/x86_64/"+pkg) {
+			t.Fatalf("%s: want 302 to %s, got %d %q", pkg, member, resp.StatusCode, loc)
+		}
+	}
+
+	script := "set -eu; " +
+		"echo \"$URL\" > /etc/apk/repositories; " +
+		"apk update --allow-untrusted; " +
+		"apk add --allow-untrusted vapk-one vapk-two; " +
+		"test -f /usr/share/vapk-one/ok && test -f /usr/share/vapk-two/ok"
+	out, err := exec.Command("docker", "run", "--rm", "--network", network,
+		"-e", "URL="+strings.TrimRight(internal, "/")+"/api/v1/virtual/vapk",
+		"alpine:3.20", "sh", "-c", script).CombinedOutput()
+	if err != nil {
+		t.Fatalf("real apk against alpine virtual failed: %v\n%s", err, out)
+	}
+}

e2e-docker/local_test.go

@@ -150,31 +150,15 @@ func TestLocalAlpineIndex(t *testing.T) {
 	apk := testsupport.MinimalApk("e2e-testpkg", "1.0-r0", "x86_64")
 	uploadFile(t, "local-alpine", "x86_64/e2e-testpkg-1.0-r0.apk", apk, "application/vnd.android.package-archive")
 
-	// The index is generated asynchronously after upload; poll for it.
+	// The index is served (empty) at once but lists the package only once the
-	resp, body := getEventually(t, api("/api/v1/local/local-alpine/x86_64/APKINDEX.tar.gz"), 15*time.Second)
+	// upload's metadata is parsed asynchronously; poll until it does.
-	if resp.StatusCode != http.StatusOK {
-		t.Fatalf("APKINDEX: status %d: %s", resp.StatusCode, body)
-	}
-
-	zr, err := gzip.NewReader(bytes.NewReader(body))
-	if err != nil {
-		t.Fatalf("APKINDEX not gzip: %v", err)
-	}
-	tarBytes, _ := io.ReadAll(zr)
-	tr := tar.NewReader(bytes.NewReader(tarBytes))
 	var index string
-	for {
+	for deadline := time.Now().Add(15 * time.Second); !strings.Contains(index, "P:e2e-testpkg") && time.Now().Before(deadline); time.Sleep(250 * time.Millisecond) {
-		hdr, err := tr.Next()
+		resp, body := doRequest(t, http.MethodGet, api("/api/v1/local/local-alpine/x86_64/APKINDEX.tar.gz"), nil, "")
-		if err == io.EOF {
+		if resp.StatusCode != http.StatusOK {
-			break
+			t.Fatalf("APKINDEX: status %d: %s", resp.StatusCode, body)
-		}
-		if err != nil {
-			t.Fatalf("APKINDEX not tar: %v", err)
-		}
-		if hdr.Name == "APKINDEX" {
-			b, _ := io.ReadAll(tr)
-			index = string(b)
 		}
+		index = apkIndexText(t, body)
 	}
 	for _, want := range []string{"P:e2e-testpkg", "V:1.0-r0", "A:x86_64", "C:Q1", "S:", "I:"} {
 		if !strings.Contains(index, want) {
@@ -183,7 +167,7 @@ func TestLocalAlpineIndex(t *testing.T) {
 	}
 
 	// The .apk downloads back byte-identical from its arch path.
-	resp, body = doRequest(t, http.MethodGet, api("/api/v1/local/local-alpine/x86_64/e2e-testpkg-1.0-r0.apk"), nil, "")
+	resp, body := doRequest(t, http.MethodGet, api("/api/v1/local/local-alpine/x86_64/e2e-testpkg-1.0-r0.apk"), nil, "")
 	if resp.StatusCode != http.StatusOK {
 		t.Fatalf("download apk: status %d: %s", resp.StatusCode, body)
 	}
@@ -191,3 +175,25 @@ func TestLocalAlpineIndex(t *testing.T) {
 		t.Fatalf("apk content mismatch")
 	}
 }
+
+func apkIndexText(t *testing.T, body []byte) string {
+	t.Helper()
+	zr, err := gzip.NewReader(bytes.NewReader(body))
+	if err != nil {
+		t.Fatalf("APKINDEX not gzip: %v", err)
+	}
+	tr := tar.NewReader(zr)
+	for {
+		hdr, err := tr.Next()
+		if err == io.EOF {
+			return ""
+		}
+		if err != nil {
+			t.Fatalf("APKINDEX not tar: %v", err)
+		}
+		if hdr.Name == "APKINDEX" {
+			b, _ := io.ReadAll(tr)
+			return string(b)
+		}
+	}
+}

internal/api/v1/proxy.go

@@ -2,6 +2,7 @@ package v1
 
 import (
 	"bytes"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
@@ -11,6 +12,7 @@ import (
 	"net/url"
 	"strconv"
 	"strings"
+	"time"
 
 	"github.com/go-chi/chi/v5"
 
@@ -121,7 +123,7 @@ func (h *ProxyHandler) serveFetch(w http.ResponseWriter, r *http.Request, remote
 	if err != nil {
 		var proxyErr *proxy.ProxyError
 		if errors.As(err, &proxyErr) {
-			http.Error(w, proxyErr.Message, proxyErr.Status)
+			writeProxyError(w, proxyErr)
 			return
 		}
 		slog.Error("proxy fetch failed", "remote", remote.Name, "path", path, "error", err)
@@ -149,6 +151,24 @@ func (h *ProxyHandler) serveFetch(w http.ResponseWriter, r *http.Request, remote
 	io.Copy(w, body)
 }
 
+// writeProxyError answers a quarantine refusal as JSON and anything else as text.
+func writeProxyError(w http.ResponseWriter, pe *proxy.ProxyError) {
+	if pe.RetryAfter > 0 {
+		w.Header().Set("Retry-After", strconv.Itoa(pe.RetryAfter))
+	}
+	if pe.Published.IsZero() {
+		http.Error(w, pe.Message, pe.Status)
+		return
+	}
+	w.Header().Set("Content-Type", "application/json")
+	w.WriteHeader(pe.Status)
+	_ = json.NewEncoder(w).Encode(map[string]string{
+		"error":     pe.Message,
+		"published": pe.Published.Format(time.RFC3339),
+		"available": pe.Available.Format(time.RFC3339),
+	})
+}
+
 func rewritesMutable(prov provider.Provider, path string) bool {
 	_, sr := prov.(provider.ServeRewriter)
 	_, rr := prov.(provider.ResponseRewriter)
@@ -176,7 +196,7 @@ func (h *ProxyHandler) handleProxyHead(w http.ResponseWriter, r *http.Request) {
 	if err != nil {
 		var proxyErr *proxy.ProxyError
 		if errors.As(err, &proxyErr) {
-			http.Error(w, proxyErr.Message, proxyErr.Status)
+			writeProxyError(w, proxyErr)
 			return
 		}
 		slog.Error("proxy head failed", "remote", remote.Name, "path", path, "error", err)

internal/api/v1/proxyerror_test.go

@@ -0,0 +1,16 @@
+package v1
+
+import (
+	"net/http/httptest"
+	"testing"
+
+	"git.unkin.net/unkin/artifactapi/internal/proxy"
+)
+
+func TestWriteProxyErrorRetryAfter(t *testing.T) {
+	w := httptest.NewRecorder()
+	writeProxyError(w, &proxy.ProxyError{Status: 503, Message: "quarantine check unavailable", RetryAfter: 30})
+	if w.Code != 503 || w.Header().Get("Retry-After") != "30" {
+		t.Errorf("code %d, Retry-After %q", w.Code, w.Header().Get("Retry-After"))
+	}
+}

internal/api/v2/remotes.go

@@ -72,7 +72,7 @@ func (h *RemotesHandler) get(w http.ResponseWriter, r *http.Request) {
 const defaultMutableTTL = 3600
 
 func (h *RemotesHandler) create(w http.ResponseWriter, r *http.Request) {
-	remote := models.Remote{MutableTTL: defaultMutableTTL}
+	remote := models.Remote{MutableTTL: defaultMutableTTL, QuarantineDays: models.DefaultQuarantineDays}
 	if err := json.NewDecoder(r.Body).Decode(&remote); err != nil {
 		http.Error(w, "invalid json", http.StatusBadRequest)
 		return
@@ -104,6 +104,14 @@ func (h *RemotesHandler) create(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, err.Error(), http.StatusBadRequest)
 		return
 	}
+	if err := remote.ValidateSyncInterval(); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+	if err := remote.ValidateQuarantine(); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
 	if err := h.db.CreateRemote(r.Context(), &remote); err != nil {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
 		return
@@ -126,6 +134,7 @@ func (h *RemotesHandler) update(w http.ResponseWriter, r *http.Request) {
 	var remote models.Remote
 	if oldKnown {
 		remote.MutableTTL = existing.MutableTTL
+		remote.QuarantineDays = existing.QuarantineDays
 	}
 	if err := json.NewDecoder(r.Body).Decode(&remote); err != nil {
 		http.Error(w, "invalid json", http.StatusBadRequest)
@@ -144,6 +153,14 @@ func (h *RemotesHandler) update(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, err.Error(), http.StatusBadRequest)
 		return
 	}
+	if err := remote.ValidateSyncInterval(); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+	if err := remote.ValidateQuarantine(); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
 	if err := h.db.UpdateRemote(r.Context(), &remote); err != nil {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
 		return

internal/api/v2/remotes_cache_repodata_test.go

@@ -0,0 +1,57 @@
+package v2
+
+import (
+	"encoding/json"
+	"net/http/httptest"
+	"testing"
+
+	"git.unkin.net/unkin/artifactapi/internal/database"
+)
+
+func TestCacheRepodataRoundTrip(t *testing.T) {
+	if testDSN == "" {
+		t.Skip("Docker unavailable")
+	}
+	db, err := database.New(testDSN)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	h := NewRemotesHandler(db, nil, nil).Routes()
+	get := func() bool {
+		t.Helper()
+		w := httptest.NewRecorder()
+		h.ServeHTTP(w, httptest.NewRequest("GET", "/cache-rd", nil))
+		var got struct {
+			CacheRepodata *bool `json:"cache_repodata"`
+		}
+		if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil || got.CacheRepodata == nil {
+			t.Fatalf("get = %d %s", w.Code, w.Body.String())
+		}
+		return *got.CacheRepodata
+	}
+	const body = `"package_type":"github_rpm","base_url":"https://api.github.com"`
+
+	steps := []struct {
+		method, path, json string
+		want               bool
+	}{
+		{"POST", "/", `{"name":"cache-rd",` + body + `}`, false},
+		{"PUT", "/cache-rd", `{` + body + `,"cache_repodata":true}`, true},
+		{"PUT", "/cache-rd", `{` + body + `,"cache_repodata":false}`, false},
+	}
+	for i, s := range steps {
+		if c := do(t, h, s.method, s.path, s.json); c/100 != 2 {
+			t.Fatalf("step %d %s %s = %d", i, s.method, s.path, c)
+		}
+		if got := get(); got != s.want {
+			t.Errorf("step %d: cache_repodata = %v, want %v", i, got, s.want)
+		}
+	}
+	if c := do(t, h, "POST", "/", `{"name":"cache-rd-on",`+body+`,"cache_repodata":true}`); c != 201 {
+		t.Fatalf("create with cache_repodata = %d", c)
+	}
+	if r, err := db.GetRemote(t.Context(), "cache-rd-on"); err != nil || !r.CacheRepodata {
+		t.Fatalf("create did not store cache_repodata: %v %v", r, err)
+	}
+}

internal/api/v2/remotes_ttl_test.go

@@ -55,3 +55,120 @@ func TestMutableTTLOmittedVsZero(t *testing.T) {
 		t.Errorf("explicit 0 stored as %d", got)
 	}
 }
+
+// sync_interval round-trips through the API and DB on github_* remotes and is
+// rejected when negative, over the maximum, or on a type without a background syncer.
+func TestSyncIntervalAPI(t *testing.T) {
+	if testDSN == "" {
+		t.Skip("Docker unavailable")
+	}
+	db, err := database.New(testDSN)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	h := NewRemotesHandler(db, nil, nil).Routes()
+	interval := func(name string) int {
+		t.Helper()
+		r, err := db.GetRemote(context.Background(), name)
+		if err != nil {
+			t.Fatalf("get %s: %v", name, err)
+		}
+		return r.SyncInterval
+	}
+	const gh = `"package_type":"github_deb","base_url":"https://api.github.com/repos/acme/tools","releases_remote":"github"`
+
+	if c := do(t, h, "POST", "/", `{"name":"si-gh",`+gh+`}`); c != 201 {
+		t.Fatalf("create unset = %d", c)
+	}
+	if got := interval("si-gh"); got != 0 {
+		t.Fatalf("unset sync_interval stored as %d", got)
+	}
+	if c := do(t, h, "PUT", "/si-gh", `{`+gh+`,"sync_interval":3600}`); c != 200 {
+		t.Fatalf("update = %d", c)
+	}
+	if got := interval("si-gh"); got != 3600 {
+		t.Fatalf("sync_interval = %d, want 3600", got)
+	}
+
+	for _, body := range []string{
+		`{"name":"si-neg",` + gh + `,"sync_interval":-1}`,
+		`{"name":"si-max",` + gh + `,"sync_interval":2592001}`,
+		`{"name":"si-int32",` + gh + `,"sync_interval":2147483648}`,
+		`{"name":"si-rpm","package_type":"rpm","base_url":"https://example.com/repo","sync_interval":60}`,
+	} {
+		if c := do(t, h, "POST", "/", body); c != 400 {
+			t.Errorf("%s = %d, want 400", body, c)
+		}
+	}
+	if c := do(t, h, "PUT", "/si-gh", `{`+gh+`,"sync_interval":-1}`); c != 400 {
+		t.Errorf("update negative = %d, want 400", c)
+	}
+	if got := interval("si-gh"); got != 3600 {
+		t.Errorf("rejected update changed sync_interval to %d", got)
+	}
+}
+
+func TestQuarantineDaysDefault(t *testing.T) {
+	if testDSN == "" {
+		t.Skip("Docker unavailable")
+	}
+	db, err := database.New(testDSN)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	h := NewRemotesHandler(db, nil, nil).Routes()
+	days := func() int {
+		t.Helper()
+		r, err := db.GetRemote(context.Background(), "q-days")
+		if err != nil {
+			t.Fatal(err)
+		}
+		return r.QuarantineDays
+	}
+	const body = `"package_type":"npm","base_url":"https://example.com","quarantine_enabled":true`
+	for i, s := range []struct {
+		method, path, json string
+		want               int
+	}{
+		{"POST", "/", `{"name":"q-days",` + body + `}`, 3},
+		{"PUT", "/q-days", `{` + body + `,"quarantine_days":14}`, 14},
+		{"PUT", "/q-days", `{` + body + `}`, 14},
+	} {
+		if c := do(t, h, s.method, s.path, s.json); c/100 != 2 {
+			t.Fatalf("step %d = %d", i, c)
+		}
+		if got := days(); got != s.want {
+			t.Errorf("step %d: quarantine_days = %d, want %d", i, got, s.want)
+		}
+	}
+}
+
+func TestQuarantineDaysValidation(t *testing.T) {
+	if testDSN == "" {
+		t.Skip("Docker unavailable")
+	}
+	db, err := database.New(testDSN)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	h := NewRemotesHandler(db, nil, nil).Routes()
+	const body = `"package_type":"npm","base_url":"https://example.com"`
+	for i, s := range []struct {
+		method, path, json string
+		want               int
+	}{
+		{"POST", "/", `{"name":"q-val",` + body + `,"quarantine_enabled":true,"quarantine_days":0}`, 400},
+		{"POST", "/", `{"name":"q-val",` + body + `,"quarantine_enabled":true,"quarantine_days":-1}`, 400},
+		{"POST", "/", `{"name":"q-val",` + body + `,"quarantine_days":0}`, 201},
+		{"PUT", "/q-val", `{` + body + `,"quarantine_enabled":true}`, 400},
+		{"PUT", "/q-val", `{` + body + `,"quarantine_enabled":true,"quarantine_days":1}`, 200},
+		{"PUT", "/q-val", `{` + body + `,"quarantine_enabled":true,"quarantine_days":0}`, 400},
+	} {
+		if c := do(t, h, s.method, s.path, s.json); c != s.want {
+			t.Errorf("step %d %s %s = %d, want %d", i, s.method, s.path, c, s.want)
+		}
+	}
+}

internal/api/v2/virtuals_test.go

@@ -0,0 +1,56 @@
+package v2
+
+import (
+	"encoding/json"
+	"net/http/httptest"
+	"testing"
+
+	"git.unkin.net/unkin/artifactapi/internal/database"
+)
+
+func TestVirtualParallelFetchRoundTrip(t *testing.T) {
+	if testDSN == "" {
+		t.Skip("Docker unavailable")
+	}
+	db, err := database.New(testDSN)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	h := NewVirtualsHandler(db).Routes()
+	parallel := func() bool {
+		t.Helper()
+		w := httptest.NewRecorder()
+		h.ServeHTTP(w, httptest.NewRequest("GET", "/vpf", nil))
+		var got map[string]any
+		if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil || w.Code != 200 {
+			t.Fatalf("get = %d %s", w.Code, w.Body)
+		}
+		v, ok := got["parallel_fetch"].(bool)
+		if !ok {
+			t.Fatalf("parallel_fetch missing from %s", w.Body)
+		}
+		return v
+	}
+	const body = `"package_type":"rpm","members":["a"]`
+
+	steps := []struct {
+		method, path, json string
+		want               bool
+	}{
+		{"POST", "/", `{"name":"vpf",` + body + `}`, false},
+		{"PUT", "/vpf", `{` + body + `,"parallel_fetch":true}`, true},
+		{"PUT", "/vpf", `{` + body + `,"parallel_fetch":false}`, false},
+	}
+	for i, s := range steps {
+		if c := do(t, h, s.method, s.path, s.json); c/100 != 2 {
+			t.Fatalf("step %d %s %s = %d", i, s.method, s.path, c)
+		}
+		if got := parallel(); got != s.want {
+			t.Errorf("step %d: parallel_fetch = %v, want %v", i, got, s.want)
+		}
+	}
+	if c := do(t, h, "DELETE", "/vpf", ""); c != 204 {
+		t.Fatalf("delete = %d", c)
+	}
+}

internal/auth/oidc.go

@@ -2,7 +2,9 @@ package auth
 
 import (
 	"context"
+	"encoding/json"
 	"errors"
+	"fmt"
 	"log/slog"
 	"net/http"
 	"slices"
@@ -104,42 +106,116 @@ func (b *BearerOIDC) discoverOnce(done chan struct{}) (v *oidc.IDTokenVerifier,
 // Require gates every request through next on a valid token. A nil receiver
 // passes everything through.
 func (b *BearerOIDC) Require(next http.Handler) http.Handler {
+	return b.require(next, "Bearer")
+}
+
+func (b *BearerOIDC) require(next http.Handler, challenge string) http.Handler {
 	if b == nil {
 		return next
 	}
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		scheme, raw, _ := strings.Cut(strings.TrimSpace(r.Header.Get("Authorization")), " ")
+		raw := token(r)
-		raw = strings.TrimSpace(raw)
+		if raw == "" {
-		if !strings.EqualFold(scheme, "Bearer") || raw == "" {
+			unauthorized(w, challenge, "missing token")
-			unauthorized(w, "missing bearer token")
 			return
 		}
-		v, err := b.getVerifier(r.Context())
+		sub, err := b.verify(r.Context(), raw)
-		if err != nil {
+		if errors.Is(err, errIssuerUnavailable) {
 			w.Header().Set("Content-Type", "application/json")
 			w.WriteHeader(http.StatusServiceUnavailable)
 			_, _ = w.Write([]byte(`{"error":"token issuer unavailable"}`))
 			return
 		}
-		tok, err := v.Verify(r.Context(), raw)
 		if err != nil {
 			slog.Info("bearer token rejected", "error", err)
-			unauthorized(w, "invalid token")
+			unauthorized(w, challenge, "invalid token")
 			return
 		}
-		if len(b.audiences) > 0 && !slices.ContainsFunc(tok.Audience, func(a string) bool { return slices.Contains(b.audiences, a) }) {
+		slog.Info("bearer token accepted", "sub", sub, "method", r.Method, "path", r.URL.Path)
-			slog.Info("bearer token rejected", "error", "audience not allowed", "aud", tok.Audience)
-			unauthorized(w, "invalid token")
-			return
-		}
-		slog.Info("bearer token accepted", "sub", tok.Subject, "method", r.Method, "path", r.URL.Path)
 		next.ServeHTTP(w, r)
 	})
 }
 
+func (b *BearerOIDC) verify(ctx context.Context, raw string) (string, error) {
+	v, err := b.getVerifier(ctx)
+	if err != nil {
+		return "", errIssuerUnavailable
+	}
+	tok, err := v.Verify(ctx, raw)
+	if err != nil {
+		return "", err
+	}
+	if len(b.audiences) > 0 && !slices.ContainsFunc(tok.Audience, func(a string) bool { return slices.Contains(b.audiences, a) }) {
+		return "", fmt.Errorf("audience %v not allowed", tok.Audience)
+	}
+	return tok.Subject, nil
+}
+
+// token reads the JWT from `Bearer`/`Token` credentials, the password of
+// `Basic` credentials (any username), or a bare header value as `gem push`
+// sends its API key.
+func token(r *http.Request) string {
+	if _, pass, ok := r.BasicAuth(); ok {
+		return pass
+	}
+	scheme, raw, found := strings.Cut(strings.TrimSpace(r.Header.Get("Authorization")), " ")
+	if !found {
+		return scheme
+	}
+	if strings.EqualFold(scheme, "Bearer") || strings.EqualFold(scheme, "Token") {
+		return strings.TrimSpace(raw)
+	}
+	return ""
+}
+
 // RequireWrites gates only mutating methods.
 func (b *BearerOIDC) RequireWrites(next http.Handler) http.Handler {
-	gated := b.Require(next)
+	return writesOnly(next, b.Require(next))
+}
+
+// RequireRegistryWrites gates Docker Registry V2 writes behind the token flow
+// Docker clients need: an unauthenticated ping gets a Bearer challenge naming
+// realm, which serves the Basic password back as the token once verified, or
+// an anonymous token so pulls stay open.
+func (b *BearerOIDC) RequireRegistryWrites(next http.Handler, realm func(*http.Request) string) http.Handler {
+	if b == nil {
+		return next
+	}
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		challenge := `Bearer realm="` + realm(r) + `",service="artifactapi"`
+		hasAuth := r.Header.Get("Authorization") != ""
+		switch {
+		case r.URL.Path == "/v2/token":
+			writeToken(w, b.registryToken(r))
+		case r.URL.Path == "/v2/" && !hasAuth:
+			unauthorized(w, challenge, "authentication required")
+		default:
+			writesOnly(next, b.require(next, challenge)).ServeHTTP(w, r)
+		}
+	})
+}
+
+// registryToken echoes a valid credential back as the registry token. Missing
+// or invalid credentials (e.g. an expired `docker login`) get the anonymous
+// token so pulls keep working; writes are still verified per request.
+func (b *BearerOIDC) registryToken(r *http.Request) string {
+	raw := token(r)
+	if raw == "" {
+		return "anonymous"
+	}
+	if _, err := b.verify(r.Context(), raw); err != nil {
+		slog.Info("registry credential rejected, issuing anonymous token", "error", err)
+		return "anonymous"
+	}
+	return raw
+}
+
+func writeToken(w http.ResponseWriter, tok string) {
+	w.Header().Set("Content-Type", "application/json")
+	_ = json.NewEncoder(w).Encode(map[string]string{"token": tok, "access_token": tok})
+}
+
+func writesOnly(next, gated http.Handler) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		switch r.Method {
 		case http.MethodGet, http.MethodHead, http.MethodOptions:
@@ -150,9 +226,9 @@ func (b *BearerOIDC) RequireWrites(next http.Handler) http.Handler {
 	})
 }
 
-func unauthorized(w http.ResponseWriter, msg string) {
+func unauthorized(w http.ResponseWriter, challenge, msg string) {
 	w.Header().Set("Content-Type", "application/json")
-	w.Header().Set("WWW-Authenticate", "Bearer")
+	w.Header().Set("WWW-Authenticate", challenge)
 	w.WriteHeader(http.StatusUnauthorized)
 	_, _ = w.Write([]byte(`{"error":"` + msg + `"}`))
 }

internal/auth/oidc_test.go

@@ -8,6 +8,7 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"strconv"
+	"strings"
 	"sync"
 	"sync/atomic"
 	"testing"
@@ -253,3 +254,90 @@ func TestRequireWritesLeavesReadsOpen(t *testing.T) {
 		}
 	}
 }
+
+func basic(user, pass string) string {
+	return "Basic " + base64.StdEncoding.EncodeToString([]byte(user+":"+pass))
+}
+
+func TestTokenSchemes(t *testing.T) {
+	iss := testsupport.NewOIDCIssuer()
+	defer iss.Close()
+	h := NewBearerOIDC(iss.URL, []string{"artifactapi"}).RequireWrites(ok)
+	valid := iss.Token(nil, "artifactapi", "ci", time.Hour)
+	cases := []struct {
+		name  string
+		authz string
+		want  int
+	}{
+		{"bearer", "Bearer " + valid, http.StatusNoContent},
+		{"token scheme", "Token " + valid, http.StatusNoContent},
+		{"basic password", basic("anyone", valid), http.StatusNoContent},
+		{"basic empty user", basic("", valid), http.StatusNoContent},
+		{"bare jwt", valid, http.StatusNoContent},
+		{"basic wrong aud", basic("ci", iss.Token(nil, "nope", "ci", time.Hour)), http.StatusUnauthorized},
+		{"basic token as user", basic(valid, "x"), http.StatusUnauthorized},
+		{"bare garbage", "garbage", http.StatusUnauthorized},
+		{"unknown scheme", "Digest " + valid, http.StatusUnauthorized},
+	}
+	for _, c := range cases {
+		if got := serveAuth(h, http.MethodPut, c.authz); got != c.want {
+			t.Errorf("%s: status %d, want %d", c.name, got, c.want)
+		}
+	}
+}
+
+func TestRequireRegistryWrites(t *testing.T) {
+	iss := testsupport.NewOIDCIssuer()
+	defer iss.Close()
+	h := NewBearerOIDC(iss.URL, []string{"artifactapi"}).RequireRegistryWrites(ok, func(*http.Request) string { return "https://reg.example/v2/token" })
+	valid := iss.Token(nil, "artifactapi", "ci", time.Hour)
+	do := func(method, path, authz string) *httptest.ResponseRecorder {
+		r := httptest.NewRequest(method, path, nil)
+		if authz != "" {
+			r.Header.Set("Authorization", authz)
+		}
+		w := httptest.NewRecorder()
+		h.ServeHTTP(w, r)
+		return w
+	}
+	challenge := `Bearer realm="https://reg.example/v2/token",service="artifactapi"`
+
+	w := do(http.MethodGet, "/v2/", "")
+	if w.Code != http.StatusUnauthorized || w.Header().Get("WWW-Authenticate") != challenge {
+		t.Errorf("anonymous ping: %d %q", w.Code, w.Header().Get("WWW-Authenticate"))
+	}
+	if w := do(http.MethodGet, "/v2/", "Bearer anonymous"); w.Code != http.StatusNoContent {
+		t.Errorf("ping with token: %d", w.Code)
+	}
+	if w := do(http.MethodGet, "/v2/token", ""); w.Code != http.StatusOK || !strings.Contains(w.Body.String(), `"token":"anonymous"`) {
+		t.Errorf("anonymous token: %d %s", w.Code, w.Body)
+	}
+	if w := do(http.MethodGet, "/v2/token", basic("ci", valid)); w.Code != http.StatusOK || !strings.Contains(w.Body.String(), `"token":"`+valid+`"`) {
+		t.Errorf("token exchange: %d %s", w.Code, w.Body)
+	}
+	expired := iss.Token(nil, "artifactapi", "ci", -time.Minute)
+	for _, authz := range []string{basic("ci", "bogus"), basic("ci", expired), "Bearer " + expired} {
+		if w := do(http.MethodGet, "/v2/token", authz); w.Code != http.StatusOK || !strings.Contains(w.Body.String(), `"token":"anonymous"`) {
+			t.Errorf("rejected credential %q: %d %s", authz, w.Code, w.Body)
+		}
+	}
+	for _, m := range []string{http.MethodGet, http.MethodHead} {
+		if w := do(m, "/v2/repo/img/manifests/t", ""); w.Code != http.StatusNoContent {
+			t.Errorf("anonymous %s: %d", m, w.Code)
+		}
+	}
+	for _, m := range []string{http.MethodPost, http.MethodPatch, http.MethodPut, http.MethodDelete} {
+		w := do(m, "/v2/repo/img/blobs/uploads/", "Bearer anonymous")
+		if w.Code != http.StatusUnauthorized || w.Header().Get("WWW-Authenticate") != challenge {
+			t.Errorf("anonymous %s: %d %q", m, w.Code, w.Header().Get("WWW-Authenticate"))
+		}
+		for _, authz := range []string{"Bearer " + valid, basic("ci", valid)} {
+			if w := do(m, "/v2/repo/img/blobs/uploads/", authz); w.Code != http.StatusNoContent {
+				t.Errorf("authenticated %s: %d", m, w.Code)
+			}
+		}
+	}
+	if got := serve(NewBearerOIDC("", nil).RequireRegistryWrites(ok, nil), http.MethodPut, ""); got != http.StatusNoContent {
+		t.Errorf("disabled: status %d", got)
+	}
+}

internal/cache/redis.go

@@ -84,6 +84,20 @@ func (r *Redis) GetETag(ctx context.Context, remote, path string) (string, error
 	return val, err
 }
 
+// FirstSeen records now as the first sighting of path unless one is already
+// stored, and returns the stored sighting.
+func (r *Redis) FirstSeen(ctx context.Context, remote, path string, ttl time.Duration) (time.Time, error) {
+	key := fmt.Sprintf("firstseen:%s:%s", remote, path)
+	if err := r.client.SetNX(ctx, key, time.Now().Unix(), ttl).Err(); err != nil {
+		return time.Time{}, err
+	}
+	sec, err := r.client.Get(ctx, key).Int64()
+	if err != nil {
+		return time.Time{}, err
+	}
+	return time.Unix(sec, 0), nil
+}
+
 func (r *Redis) GetToken(ctx context.Context, key string) (string, error) {
 	val, err := r.client.Get(ctx, "token:"+key).Result()
 	if err == redis.Nil {

internal/config/env.go

@@ -61,6 +61,8 @@ type Config struct {
 	// Bearer-JWT auth on configuration writes; empty issuer disables it.
 	OIDCIssuer    string
 	OIDCAudiences []string
+	// AuthUploads extends that auth to local repo uploads, deletes and evictions.
+	AuthUploads bool
 }
 
 func (c *Config) DatabaseDSN() string {
@@ -105,6 +107,13 @@ func Load() (*Config, error) {
 	if oidcIssuer != "" && len(oidcAudiences) == 0 {
 		return nil, fmt.Errorf("ARTIFACTAPI_OIDC_AUDIENCE is required when ARTIFACTAPI_OIDC_ISSUER is set")
 	}
+	authUploads, err := strconv.ParseBool(getenv("ARTIFACTAPI_AUTH_UPLOADS", "false"))
+	if err != nil {
+		return nil, fmt.Errorf("invalid ARTIFACTAPI_AUTH_UPLOADS: %w", err)
+	}
+	if authUploads && oidcIssuer == "" {
+		return nil, fmt.Errorf("ARTIFACTAPI_OIDC_ISSUER is required when ARTIFACTAPI_AUTH_UPLOADS is set")
+	}
 
 	cfg := &Config{
 		ListenAddr: getenv("LISTEN_ADDR", ":8000"),
@@ -142,6 +151,7 @@ func Load() (*Config, error) {
 
 		OIDCIssuer:    oidcIssuer,
 		OIDCAudiences: oidcAudiences,
+		AuthUploads:   authUploads,
 	}
 
 	return cfg, nil

internal/config/env_test.go

@@ -81,3 +81,23 @@ func TestLoadOIDC(t *testing.T) {
 		t.Error("expected error for issuer without audience")
 	}
 }
+
+func TestLoadAuthUploads(t *testing.T) {
+	cfg, err := Load()
+	if err != nil || cfg.AuthUploads {
+		t.Fatalf("default: %v, AuthUploads=%v", err, cfg.AuthUploads)
+	}
+	t.Setenv("ARTIFACTAPI_AUTH_UPLOADS", "true")
+	if _, err := Load(); err == nil {
+		t.Error("expected error for AUTH_UPLOADS without issuer")
+	}
+	t.Setenv("ARTIFACTAPI_OIDC_ISSUER", "https://vault.example/v1/identity/oidc")
+	t.Setenv("ARTIFACTAPI_OIDC_AUDIENCE", "artifactapi")
+	if cfg, err := Load(); err != nil || !cfg.AuthUploads {
+		t.Fatalf("enabled: %v, AuthUploads=%v", err, cfg.AuthUploads)
+	}
+	t.Setenv("ARTIFACTAPI_AUTH_UPLOADS", "maybe")
+	if _, err := Load(); err == nil {
+		t.Error("expected error for invalid ARTIFACTAPI_AUTH_UPLOADS")
+	}
+}

internal/database/alpine_github_sync.go

@@ -30,8 +30,8 @@ func (db *DB) ListGitHubAlpineRemotes(ctx context.Context) ([]models.Remote, err
 
 // ClaimGitHubAlpineSyncLease atomically claims the per-remote github_alpine sync
 // lease. See claimSyncLease.
-func (db *DB) ClaimGitHubAlpineSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (db *DB) ClaimGitHubAlpineSyncLease(ctx context.Context, remoteName, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
-	return db.claimSyncLease(ctx, "github_alpine_sync_state", remoteName, owner, freshness, lease)
+	return db.claimSyncLease(ctx, "github_alpine_sync_state", remoteName, owner, freshness, lease, interval, phase)
 }
 
 // ReleaseGitHubAlpineSyncLease records a github_alpine scan outcome and frees the

internal/database/database_test.go

@@ -79,13 +79,19 @@ func TestRemotesCRUD(t *testing.T) {
 	if err != nil || got.BaseURL != "https://example.com" {
 		t.Fatalf("get: %v %v", got, err)
 	}
+	if got.CacheRepodata {
+		t.Error("cache_repodata should default to false")
+	}
 	got.BaseURL = "https://updated.example.com"
+	got.CacheRepodata = true
+	got.SyncInterval = 600
+	got.ServeStale = true
 	if err := testDB.UpdateRemote(ctx(), got); err != nil {
 		t.Fatalf("update: %v", err)
 	}
 	got, _ = testDB.GetRemote(ctx(), "r-crud")
-	if got.BaseURL != "https://updated.example.com" {
+	if got.BaseURL != "https://updated.example.com" || !got.CacheRepodata || got.SyncInterval != 600 || !got.ServeStale {
-		t.Errorf("update not applied: %v", got.BaseURL)
+		t.Errorf("update not applied: %v %v %v %v", got.BaseURL, got.CacheRepodata, got.SyncInterval, got.ServeStale)
 	}
 	list, err := testDB.ListRemotes(ctx())
 	if err != nil || len(list) == 0 {
@@ -307,6 +313,68 @@ func TestLocalFiles(t *testing.T) {
 	}
 }
 
+// A repo recreated under a deleted repo's name must not inherit its files or
+// metadata, or re-uploading the same path 409s.
+func TestDeleteRemoteDropsLocalFiles(t *testing.T) {
+	requireDB(t)
+	seedRemote(t, "r-recreate")
+	hash := seedBlob(t, "recreatehash")
+	if err := testDB.CreateLocalFile(ctx(), "r-recreate", "x86_64/a-1.0-r0.apk", hash); err != nil {
+		t.Fatal(err)
+	}
+	if err := testDB.InsertAlpineMetadata(ctx(), &provider.AlpineMetadata{RepoName: "r-recreate", FilePath: "x86_64/a-1.0-r0.apk", ContentHash: hash, Name: "a", Version: "1.0-r0", Arch: "x86_64"}); err != nil {
+		t.Fatal(err)
+	}
+	if err := testDB.DeleteRemote(ctx(), "r-recreate"); err != nil {
+		t.Fatal(err)
+	}
+	seedRemote(t, "r-recreate")
+	defer testDB.DeleteRemote(ctx(), "r-recreate")
+	if f, err := testDB.GetLocalFile(ctx(), "r-recreate", "x86_64/a-1.0-r0.apk"); err != nil || f != nil {
+		t.Fatalf("recreated repo inherited local file: %v %v", f, err)
+	}
+	if rows, err := testDB.ListAlpineMetadataEntries(ctx(), "r-recreate"); err != nil || len(rows) != 0 {
+		t.Fatalf("recreated repo inherited alpine metadata: %v %v", rows, err)
+	}
+	if err := testDB.CreateLocalFile(ctx(), "r-recreate", "x86_64/a-1.0-r0.apk", hash); err != nil {
+		t.Fatalf("re-upload after recreate: %v", err)
+	}
+}
+
+func TestVirtualParallelFetchRoundTrip(t *testing.T) {
+	requireDB(t)
+	if err := testDB.CreateVirtual(ctx(), &models.Virtual{
+		Name: "v-parallel", PackageType: models.PackageRPM, Members: []string{"a"}, ParallelFetch: true,
+	}); err != nil {
+		t.Fatal(err)
+	}
+	defer func() { _ = testDB.DeleteVirtual(ctx(), "v-parallel") }()
+	v, err := testDB.GetVirtual(ctx(), "v-parallel")
+	if err != nil || !v.ParallelFetch {
+		t.Fatalf("created parallel_fetch = %v, %v; want true", v, err)
+	}
+	v.ParallelFetch = false
+	if err := testDB.UpdateVirtual(ctx(), v); err != nil {
+		t.Fatal(err)
+	}
+	vs, err := testDB.ListVirtuals(ctx())
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, l := range vs {
+		if l.Name == "v-parallel" && l.ParallelFetch {
+			t.Fatal("updated parallel_fetch not persisted")
+		}
+	}
+	if err := testDB.CreateVirtual(ctx(), &models.Virtual{Name: "v-serial", PackageType: models.PackageRPM, Members: []string{"a"}}); err != nil {
+		t.Fatal(err)
+	}
+	defer func() { _ = testDB.DeleteVirtual(ctx(), "v-serial") }()
+	if v, err := testDB.GetVirtual(ctx(), "v-serial"); err != nil || v.ParallelFetch {
+		t.Fatalf("default parallel_fetch = %v, %v; want false", v, err)
+	}
+}
+
 func TestVirtualsCRUD(t *testing.T) {
 	requireDB(t)
 	if err := testDB.CreateVirtual(ctx(), &models.Virtual{
@@ -425,3 +493,44 @@ func TestRPMMetadata(t *testing.T) {
 		t.Fatalf("list rpm rows: %v %v", len(rows), err)
 	}
 }
+
+func TestRPMMetadataFingerprint(t *testing.T) {
+	requireDB(t)
+	seedRemote(t, "r-rpm-fp")
+	fp := func() [2]int64 {
+		t.Helper()
+		c, m, err := testDB.RPMMetadataFingerprint(ctx(), "r-rpm-fp")
+		if err != nil {
+			t.Fatal(err)
+		}
+		return [2]int64{c, m}
+	}
+	insert := func(path string) {
+		t.Helper()
+		if err := testDB.InsertRPMMetadata(ctx(), &provider.RPMMetadata{
+			RepoName: "r-rpm-fp", FilePath: path, ContentHash: "sha256:" + path,
+			Name: path, Version: "1", Release: "1", Arch: "noarch",
+		}); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	if got := fp(); got != [2]int64{0, 0} {
+		t.Fatalf("empty fingerprint = %v", got)
+	}
+	insert("a.rpm")
+	one := fp()
+	insert("a.rpm")
+	if fp() != one {
+		t.Fatal("conflicting insert changed fingerprint")
+	}
+	insert("b.rpm")
+	two := fp()
+	if err := testDB.DeleteRPMMetadata(ctx(), "r-rpm-fp", "a.rpm"); err != nil {
+		t.Fatal(err)
+	}
+	afterDelete := fp()
+	if one[0] != 1 || two[0] != 2 || afterDelete[0] != 1 || afterDelete == one || afterDelete == two {
+		t.Fatalf("fingerprints did not track changes: %v %v %v", one, two, afterDelete)
+	}
+}

internal/database/deb_github_sync.go

@@ -30,8 +30,8 @@ func (db *DB) ListGitHubDebRemotes(ctx context.Context) ([]models.Remote, error)
 
 // ClaimGitHubDebSyncLease atomically claims the per-remote github_deb sync
 // lease. See claimSyncLease.
-func (db *DB) ClaimGitHubDebSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (db *DB) ClaimGitHubDebSyncLease(ctx context.Context, remoteName, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
-	return db.claimSyncLease(ctx, "github_deb_sync_state", remoteName, owner, freshness, lease)
+	return db.claimSyncLease(ctx, "github_deb_sync_state", remoteName, owner, freshness, lease, interval, phase)
 }
 
 // ReleaseGitHubDebSyncLease records a github_deb scan outcome and frees the

internal/database/deb_github_sync_test.go

@@ -29,7 +29,7 @@ func TestGitHubDebSyncLease(t *testing.T) {
 	const lease = 15 * time.Minute
 	freshness := time.Hour
 
-	claimed, etag, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-1", freshness, lease)
+	claimed, etag, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-1", freshness, lease, 0, 0)
 	if err != nil || !claimed {
 		t.Fatalf("replica-1 first claim: claimed=%v err=%v", claimed, err)
 	}
@@ -37,7 +37,7 @@ func TestGitHubDebSyncLease(t *testing.T) {
 		t.Fatalf("initial etag should be empty, got %q", etag)
 	}
 
-	claimed2, _, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-2", freshness, lease)
+	claimed2, _, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-2", freshness, lease, 0, 0)
 	if err != nil {
 		t.Fatalf("replica-2 claim err: %v", err)
 	}
@@ -49,7 +49,7 @@ func TestGitHubDebSyncLease(t *testing.T) {
 		t.Fatalf("release: %v", err)
 	}
 
-	claimed3, _, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-2", freshness, lease)
+	claimed3, _, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-2", freshness, lease, 0, 0)
 	if err != nil {
 		t.Fatalf("replica-2 recency claim err: %v", err)
 	}
@@ -57,7 +57,7 @@ func TestGitHubDebSyncLease(t *testing.T) {
 		t.Fatal("periodic claim succeeded inside the freshness window")
 	}
 
-	claimed4, etag4, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-2", 0, lease)
+	claimed4, etag4, err := testDB.ClaimGitHubDebSyncLease(ctx(), name, "replica-2", 0, lease, 0, 0)
 	if err != nil || !claimed4 {
 		t.Fatalf("prime claim: claimed=%v err=%v", claimed4, err)
 	}

internal/database/github_sync.go

@@ -34,8 +34,8 @@ func (db *DB) ListGitHubRPMRemotes(ctx context.Context) ([]models.Remote, error)
 
 // ClaimGitHubSyncLease atomically claims the per-remote github_rpm sync lease.
 // See claimSyncLease.
-func (db *DB) ClaimGitHubSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (db *DB) ClaimGitHubSyncLease(ctx context.Context, remoteName, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
-	return db.claimSyncLease(ctx, "github_rpm_sync_state", remoteName, owner, freshness, lease)
+	return db.claimSyncLease(ctx, "github_rpm_sync_state", remoteName, owner, freshness, lease, interval, phase)
 }
 
 // ReleaseGitHubSyncLease records a github_rpm scan outcome and frees the lease.
@@ -48,22 +48,32 @@ func (db *DB) ReleaseGitHubSyncLease(ctx context.Context, remoteName, owner stri
 // (claimed=true) only when the remote is due and no live lease is held by
 // another replica. Due means: a pending retry after a failed scan has reached
 // next_retry_at, or, with no retry pending, the remote was never synced or was
-// synced longer than freshness ago. A zero freshness (prime scans) ignores the
+// last synced before the current slot (interval set) or longer than freshness
-// recency gate but still honours a pending retry's backoff. The returned etag is
+// ago (otherwise). With interval set, slots start at phase past each multiple
-// the stored releases-list ETag, shared across replicas so a conditional request
+// of interval and a remote created inside the current slot with no state row
+// waits for the next.
+// Slots are computed from the DB clock so replicas agree whatever their own
+// clocks say. A zero freshness and interval (prime scans) ignores the recency
+// gate but still honours a pending retry's backoff. The returned etag is the
+// stored releases-list ETag, shared across replicas so a conditional request
 // can short-circuit an unchanged repo.
-func (db *DB) claimSyncLease(ctx context.Context, table, remoteName, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (db *DB) claimSyncLease(ctx context.Context, table, remoteName, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
+	const slot = `to_timestamp(floor((extract(epoch FROM now())::float8 - $6::float8) / NULLIF($5::float8, 0)) * $5::float8 + $6::float8)`
 	row := db.Pool.QueryRow(ctx, `
 		INSERT INTO `+table+` AS s (remote_name, sync_lease_owner, sync_lease_expires)
-		VALUES ($1, $2, now() + make_interval(secs => $4))
+		SELECT r.name, $2, now() + make_interval(secs => $4)
+		  FROM remotes r
+		 WHERE r.name = $1
+		   AND ($5::float8 <= 0 OR COALESCE(r.created_at, '-infinity') < `+slot+`
+		        OR EXISTS (SELECT 1 FROM `+table+` e WHERE e.remote_name = r.name))
 		ON CONFLICT (remote_name) DO UPDATE
 			SET sync_lease_owner   = $2,
 			    sync_lease_expires = now() + make_interval(secs => $4)
 			WHERE (CASE WHEN s.next_retry_at IS NOT NULL THEN s.next_retry_at <= now()
-			            ELSE s.last_synced_at IS NULL OR s.last_synced_at < now() - make_interval(secs => $3) END)
+			            ELSE s.last_synced_at IS NULL OR s.last_synced_at < COALESCE(`+slot+`, now() - make_interval(secs => $3)) END)
 			  AND (s.sync_lease_expires IS NULL OR s.sync_lease_expires < now())
 		RETURNING s.etag
-	`, remoteName, owner, freshness.Seconds(), lease.Seconds())
+	`, remoteName, owner, freshness.Seconds(), lease.Seconds(), interval.Seconds(), phase.Seconds())
 
 	var etag string
 	if err := row.Scan(&etag); err != nil {

internal/database/github_sync_test.go

@@ -1,6 +1,10 @@
 package database
 
 import (
+	"context"
+	"fmt"
+	"math"
+	"sync/atomic"
 	"testing"
 	"time"
 
@@ -30,7 +34,7 @@ func TestGitHubSyncLease(t *testing.T) {
 	freshness := time.Hour
 
 	// First claim on a never-synced remote wins; etag starts empty.
-	claimed, etag, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-1", freshness, lease)
+	claimed, etag, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-1", freshness, lease, 0, 0)
 	if err != nil || !claimed {
 		t.Fatalf("replica-1 first claim: claimed=%v err=%v", claimed, err)
 	}
@@ -39,7 +43,7 @@ func TestGitHubSyncLease(t *testing.T) {
 	}
 
 	// A second replica cannot claim while the lease is held.
-	claimed2, _, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", freshness, lease)
+	claimed2, _, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", freshness, lease, 0, 0)
 	if err != nil {
 		t.Fatalf("replica-2 claim err: %v", err)
 	}
@@ -53,7 +57,7 @@ func TestGitHubSyncLease(t *testing.T) {
 	}
 
 	// A periodic re-claim inside the freshness window is blocked by recency.
-	claimed3, _, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", freshness, lease)
+	claimed3, _, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", freshness, lease, 0, 0)
 	if err != nil {
 		t.Fatalf("replica-2 recency claim err: %v", err)
 	}
@@ -62,7 +66,7 @@ func TestGitHubSyncLease(t *testing.T) {
 	}
 
 	// A prime (freshness 0) bypasses recency and reads the persisted etag.
-	claimed4, etag4, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", 0, lease)
+	claimed4, etag4, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", 0, lease, 0, 0)
 	if err != nil || !claimed4 {
 		t.Fatalf("prime claim: claimed=%v err=%v", claimed4, err)
 	}
@@ -108,7 +112,7 @@ func TestGitHubSyncLeaseFailureBackoff(t *testing.T) {
 	freshness := time.Hour
 	claim := func(f time.Duration) (bool, string) {
 		t.Helper()
-		ok, etag, err := testDB.ClaimGitHubSyncLease(ctx(), name, "r1", f, lease)
+		ok, etag, err := testDB.ClaimGitHubSyncLease(ctx(), name, "r1", f, lease, 0, 0)
 		if err != nil {
 			t.Fatalf("claim: %v", err)
 		}
@@ -220,21 +224,21 @@ func TestGitHubSyncStateDeletedWithRemote(t *testing.T) {
 	}{
 		{"github_rpm_sync_state", models.PackageGitHubRPM,
 			func(n string) (bool, string, error) {
-				return testDB.ClaimGitHubSyncLease(ctx(), n, "r1", time.Hour, time.Minute)
+				return testDB.ClaimGitHubSyncLease(ctx(), n, "r1", time.Hour, time.Minute, 0, 0)
 			},
 			func(n string, res provider.SyncResult) error {
 				return testDB.ReleaseGitHubSyncLease(ctx(), n, "r1", res)
 			}},
 		{"github_deb_sync_state", models.PackageGitHubDeb,
 			func(n string) (bool, string, error) {
-				return testDB.ClaimGitHubDebSyncLease(ctx(), n, "r1", time.Hour, time.Minute)
+				return testDB.ClaimGitHubDebSyncLease(ctx(), n, "r1", time.Hour, time.Minute, 0, 0)
 			},
 			func(n string, res provider.SyncResult) error {
 				return testDB.ReleaseGitHubDebSyncLease(ctx(), n, "r1", res)
 			}},
 		{"github_alpine_sync_state", models.PackageGitHubAlpine,
 			func(n string) (bool, string, error) {
-				return testDB.ClaimGitHubAlpineSyncLease(ctx(), n, "r1", time.Hour, time.Minute)
+				return testDB.ClaimGitHubAlpineSyncLease(ctx(), n, "r1", time.Hour, time.Minute, 0, 0)
 			},
 			func(n string, res provider.SyncResult) error {
 				return testDB.ReleaseGitHubAlpineSyncLease(ctx(), n, "r1", res)
@@ -276,3 +280,210 @@ func TestGitHubSyncStateDeletedWithRemote(t *testing.T) {
 		})
 	}
 }
+
+type syncTable struct {
+	table string
+	pt    models.PackageType
+	claim func(name, owner string, freshness, interval, phase time.Duration) (bool, error)
+}
+
+func syncTables() []syncTable {
+	wrap := func(fn func(context.Context, string, string, time.Duration, time.Duration, time.Duration, time.Duration) (bool, string, error)) func(string, string, time.Duration, time.Duration, time.Duration) (bool, error) {
+		return func(name, owner string, freshness, interval, phase time.Duration) (bool, error) {
+			ok, _, err := fn(ctx(), name, owner, freshness, 15*time.Minute, interval, phase)
+			return ok, err
+		}
+	}
+	return []syncTable{
+		{"github_rpm_sync_state", models.PackageGitHubRPM, wrap(testDB.ClaimGitHubSyncLease)},
+		{"github_deb_sync_state", models.PackageGitHubDeb, wrap(testDB.ClaimGitHubDebSyncLease)},
+		{"github_alpine_sync_state", models.PackageGitHubAlpine, wrap(testDB.ClaimGitHubAlpineSyncLease)},
+	}
+}
+
+var syncSeq atomic.Int64
+
+// seedSynced creates a remote of tc's type created createdAgo before the DB
+// clock and, when syncedAgo >= 0, a released lease last synced syncedAgo before it.
+func seedSynced(t *testing.T, tc syncTable, createdAgo, syncedAgo time.Duration) string {
+	t.Helper()
+	name := fmt.Sprintf("sync-%s-%d", tc.table, syncSeq.Add(1))
+	if err := testDB.CreateRemote(ctx(), &models.Remote{
+		Name: name, PackageType: tc.pt, RepoType: models.RepoTypeRemote,
+		BaseURL: "https://api.github.com/repos/acme/tools", ReleasesRemote: "github",
+	}); err != nil {
+		t.Fatalf("create remote: %v", err)
+	}
+	if _, err := testDB.Pool.Exec(ctx(), `UPDATE remotes SET created_at = now() - make_interval(secs => $2) WHERE name = $1`, name, createdAgo.Seconds()); err != nil {
+		t.Fatalf("seed created_at: %v", err)
+	}
+	if syncedAgo >= 0 {
+		if _, err := testDB.Pool.Exec(ctx(), `INSERT INTO `+tc.table+` (remote_name, last_synced_at) VALUES ($1, now() - make_interval(secs => $2))`, name, syncedAgo.Seconds()); err != nil {
+			t.Fatalf("seed sync state: %v", err)
+		}
+	}
+	return name
+}
+
+func mustClaim(t *testing.T, tc syncTable, name, owner string, freshness, interval, phase time.Duration) bool {
+	t.Helper()
+	ok, err := tc.claim(name, owner, freshness, interval, phase)
+	if err != nil {
+		t.Fatalf("claim: %v", err)
+	}
+	return ok
+}
+
+const neverSynced = -1
+
+// With no interval the claim keeps the freshness-since-last-sync predicate and
+// a new remote is due at once.
+func TestSyncLeaseUnsetIntervalFreshness(t *testing.T) {
+	requireDB(t)
+	const ttl = time.Hour
+	for _, tc := range syncTables() {
+		t.Run(tc.table, func(t *testing.T) {
+			for _, c := range []struct {
+				ago       time.Duration
+				freshness time.Duration
+				due       bool
+			}{
+				{neverSynced, ttl, true},
+				{ttl + time.Minute, ttl, true},
+				{ttl - time.Minute, ttl, false},
+				{time.Second, ttl, false},
+				{time.Second, 0, true},
+			} {
+				name := seedSynced(t, tc, time.Second, c.ago)
+				if got := mustClaim(t, tc, name, "r1", c.freshness, 0, 0); got != c.due {
+					t.Errorf("synced %v ago, freshness %v: claimed=%v, want %v", c.ago, c.freshness, got, c.due)
+				}
+			}
+		})
+	}
+}
+
+// With an interval the slot comes from the DB clock: a remote is due only when
+// created and last synced before the current slot, regardless of freshness,
+// and still one replica at a time.
+func TestSyncLeaseSlot(t *testing.T) {
+	requireDB(t)
+	var epoch float64
+	if err := testDB.Pool.QueryRow(ctx(), `SELECT extract(epoch FROM now())::float8`).Scan(&epoch); err != nil {
+		t.Fatalf("db now: %v", err)
+	}
+	const interval = time.Hour
+	// Pick the phase that puts the current slot's start 30m before the DB now.
+	phase := time.Duration(math.Mod(epoch-1800, interval.Seconds()) * float64(time.Second))
+	const day = 24 * time.Hour
+	for _, tc := range syncTables() {
+		t.Run(tc.table, func(t *testing.T) {
+			for _, c := range []struct {
+				desc            string
+				createdAgo, ago time.Duration
+				freshness       time.Duration
+				due             bool
+			}{
+				{"created inside the slot", 10 * time.Second, neverSynced, 0, false},
+				{"created before the slot", 35 * time.Minute, neverSynced, 0, true},
+				{"synced inside the slot", day, 25 * time.Minute, 0, false},
+				{"synced before the slot", day, 35 * time.Minute, 10 * 365 * day, true},
+			} {
+				name := seedSynced(t, tc, c.createdAgo, c.ago)
+				if got := mustClaim(t, tc, name, "r1", c.freshness, interval, phase); got != c.due {
+					t.Errorf("%s: claimed=%v, want %v", c.desc, got, c.due)
+				}
+				if c.due && mustClaim(t, tc, name, "r2", 0, interval, phase) {
+					t.Errorf("%s: second replica claimed a held lease", c.desc)
+				}
+			}
+		})
+	}
+}
+
+// slotPhase returns the phase that puts the current interval slot's start
+// startAgo before the DB clock.
+func slotPhase(t *testing.T, interval, startAgo time.Duration) time.Duration {
+	t.Helper()
+	var epoch float64
+	if err := testDB.Pool.QueryRow(ctx(), `SELECT extract(epoch FROM now())::float8`).Scan(&epoch); err != nil {
+		t.Fatalf("db now: %v", err)
+	}
+	return time.Duration(math.Mod(epoch-startAgo.Seconds(), interval.Seconds()) * float64(time.Second))
+}
+
+func setSyncedAgo(t *testing.T, tc syncTable, name string, ago time.Duration) {
+	t.Helper()
+	if _, err := testDB.Pool.Exec(ctx(), `UPDATE `+tc.table+` SET last_synced_at = now() - make_interval(secs => $2), sync_lease_owner = '', sync_lease_expires = NULL WHERE remote_name = $1`, name, ago.Seconds()); err != nil {
+		t.Fatalf("set last_synced_at: %v", err)
+	}
+}
+
+// Toggling sync_interval on a remote with sync state neither re-syncs at once
+// nor leaves it waiting longer than one interval (or freshness) since its last
+// sync. Elapsed time is simulated by moving last_synced_at and the slot start
+// back together, relative to the DB clock.
+func TestSyncLeaseIntervalChange(t *testing.T) {
+	requireDB(t)
+	const (
+		interval  = time.Hour
+		freshness = time.Hour
+		day       = 24 * time.Hour
+	)
+	for _, tc := range syncTables() {
+		t.Run(tc.table, func(t *testing.T) {
+			t.Run("0 to N", func(t *testing.T) {
+				name := seedSynced(t, tc, day, 25*time.Minute)
+				if mustClaim(t, tc, name, "r1", freshness, 0, 0) {
+					t.Fatal("claimed inside freshness before the change")
+				}
+				if mustClaim(t, tc, name, "r1", freshness, interval, slotPhase(t, interval, 30*time.Minute)) {
+					t.Fatal("synced inside the current slot, yet claimed right after the change")
+				}
+				// 30m later the next slot opens; the last sync is 55m old.
+				setSyncedAgo(t, tc, name, 55*time.Minute)
+				if !mustClaim(t, tc, name, "r1", freshness, interval, slotPhase(t, interval, time.Second)) {
+					t.Fatal("not due at the first slot after the last sync")
+				}
+			})
+			t.Run("N to 0", func(t *testing.T) {
+				name := seedSynced(t, tc, day, 10*time.Minute)
+				if mustClaim(t, tc, name, "r1", freshness, interval, slotPhase(t, interval, 30*time.Minute)) {
+					t.Fatal("claimed inside the slot before the change")
+				}
+				if mustClaim(t, tc, name, "r1", freshness, 0, 0) {
+					t.Fatal("synced inside freshness, yet claimed right after the change")
+				}
+				setSyncedAgo(t, tc, name, freshness+time.Second)
+				if !mustClaim(t, tc, name, "r1", freshness, 0, 0) {
+					t.Fatal("not due once freshness elapsed after the change")
+				}
+			})
+		})
+	}
+}
+
+// The created_at gate only holds back the first insert of a remote created
+// inside the current slot; once a state row exists the row's own sync and
+// retry state decide.
+func TestSyncLeaseCreatedGateInsertOnly(t *testing.T) {
+	requireDB(t)
+	const interval = time.Hour
+	phase := slotPhase(t, interval, 30*time.Minute)
+	for _, tc := range syncTables() {
+		t.Run(tc.table, func(t *testing.T) {
+			fresh := seedSynced(t, tc, 10*time.Second, neverSynced)
+			if mustClaim(t, tc, fresh, "r1", 0, interval, phase) {
+				t.Fatal("remote created inside the slot claimed with no state row")
+			}
+
+			name := seedSynced(t, tc, 10*time.Second, neverSynced)
+			if _, err := testDB.Pool.Exec(ctx(), `INSERT INTO `+tc.table+` (remote_name, sync_failures, next_retry_at) VALUES ($1, 1, now() - interval '1 second')`, name); err != nil {
+				t.Fatalf("seed failed prime: %v", err)
+			}
+			if !mustClaim(t, tc, name, "r1", 0, interval, phase) {
+				t.Fatal("created_at gate blocked a remote with a state row whose retry is due")
+			}
+		})
+	}
+}

internal/database/remotes.go

@@ -3,16 +3,19 @@ package database
 import (
 	"context"
 
+	"github.com/jackc/pgx/v5"
+
 	"git.unkin.net/unkin/artifactapi/pkg/models"
 )
 
 const remoteCols = `name, package_type, repo_type, base_url, mirrorlist, mirror_strategy, description, username, password,
-	immutable_ttl, mutable_ttl, check_mutable,
+	immutable_ttl, mutable_ttl, check_mutable, sync_interval,
 	patterns, blocklist, mutable_patterns, immutable_patterns,
 	ban_tags_enabled, ban_tags,
 	quarantine_enabled, quarantine_days, stale_on_error,
 	releases_remote, managed_by,
 	upstream_dial_timeout, upstream_tls_timeout, upstream_response_header_timeout,
+	serve_stale, cache_repodata,
 	created_at, updated_at`
 
 // normalizeMirrorStrategy maps an empty strategy to the round_robin default so
@@ -27,12 +30,13 @@ func normalizeMirrorStrategy(s string) string {
 func scanRemote(scanner interface{ Scan(...any) error }, r *models.Remote) error {
 	return scanner.Scan(
 		&r.Name, &r.PackageType, &r.RepoType, &r.BaseURL, &r.Mirrorlist, &r.MirrorStrategy, &r.Description, &r.Username, &r.Password,
-		&r.ImmutableTTL, &r.MutableTTL, &r.CheckMutable,
+		&r.ImmutableTTL, &r.MutableTTL, &r.CheckMutable, &r.SyncInterval,
 		&r.Patterns, &r.Blocklist, &r.MutablePatterns, &r.ImmutablePatterns,
 		&r.BanTagsEnabled, &r.BanTags,
 		&r.QuarantineEnabled, &r.QuarantineDays, &r.StaleOnError,
 		&r.ReleasesRemote, &r.ManagedBy,
 		&r.UpstreamDialTimeout, &r.UpstreamTLSTimeout, &r.UpstreamResponseHeaderTimeout,
+		&r.ServeStale, &r.CacheRepodata,
 		&r.CreatedAt, &r.UpdatedAt,
 	)
 }
@@ -74,8 +78,8 @@ func (db *DB) CreateRemote(ctx context.Context, r *models.Remote) error {
 			quarantine_enabled, quarantine_days, stale_on_error,
 			releases_remote, managed_by,
 			upstream_dial_timeout, upstream_tls_timeout, upstream_response_header_timeout,
-			mirror_strategy
+			mirror_strategy, sync_interval, serve_stale, cache_repodata
-		) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26)
+		) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29)
 	`,
 		r.Name, r.PackageType, r.RepoType, r.BaseURL, r.Mirrorlist, r.Description, r.Username, r.Password,
 		r.ImmutableTTL, r.MutableTTL, r.CheckMutable,
@@ -84,7 +88,7 @@ func (db *DB) CreateRemote(ctx context.Context, r *models.Remote) error {
 		r.QuarantineEnabled, r.QuarantineDays, r.StaleOnError,
 		r.ReleasesRemote, r.ManagedBy,
 		r.UpstreamDialTimeout, r.UpstreamTLSTimeout, r.UpstreamResponseHeaderTimeout,
-		normalizeMirrorStrategy(r.MirrorStrategy),
+		normalizeMirrorStrategy(r.MirrorStrategy), r.SyncInterval, r.ServeStale, r.CacheRepodata,
 	)
 	return err
 }
@@ -99,7 +103,7 @@ func (db *DB) UpdateRemote(ctx context.Context, r *models.Remote) error {
 			quarantine_enabled=$17, quarantine_days=$18, stale_on_error=$19,
 			releases_remote=$20, managed_by=$21,
 			upstream_dial_timeout=$22, upstream_tls_timeout=$23, upstream_response_header_timeout=$24,
-			mirror_strategy=$26,
+			mirror_strategy=$26, sync_interval=$27, serve_stale=$28, cache_repodata=$29,
 			updated_at=NOW()
 		WHERE name=$1
 	`,
@@ -111,12 +115,21 @@ func (db *DB) UpdateRemote(ctx context.Context, r *models.Remote) error {
 		r.ReleasesRemote, r.ManagedBy,
 		r.UpstreamDialTimeout, r.UpstreamTLSTimeout, r.UpstreamResponseHeaderTimeout,
 		r.Mirrorlist,
-		normalizeMirrorStrategy(r.MirrorStrategy),
+		normalizeMirrorStrategy(r.MirrorStrategy), r.SyncInterval, r.ServeStale, r.CacheRepodata,
 	)
 	return err
 }
 
+// DeleteRemote drops a remote with its local files and package metadata, so a
+// repo recreated under the same name starts empty.
 func (db *DB) DeleteRemote(ctx context.Context, name string) error {
-	_, err := db.Pool.Exec(ctx, `DELETE FROM remotes WHERE name = $1`, name)
+	return pgx.BeginFunc(ctx, db.Pool, func(tx pgx.Tx) error {
-	return err
+		for _, table := range []string{"local_files", "rpm_metadata", "deb_metadata", "alpine_metadata", "puppet_metadata", "helm_metadata"} {
+			if _, err := tx.Exec(ctx, `DELETE FROM `+table+` WHERE repo_name = $1`, name); err != nil {
+				return err
+			}
+		}
+		_, err := tx.Exec(ctx, `DELETE FROM remotes WHERE name = $1`, name)
+		return err
+	})
 }

internal/database/rpm_metadata.go

@@ -40,6 +40,14 @@ func (db *DB) DeleteRPMMetadata(ctx context.Context, repoName, filePath string)
 	return err
 }
 
+// RPMMetadataFingerprint is a cheap change detector for a repo's rows: they are
+// only ever inserted or deleted, so (count, max id) moves whenever they change.
+// For github_rpm remotes it relies on the sync lease serialising scans.
+func (db *DB) RPMMetadataFingerprint(ctx context.Context, repoName string) (count, maxID int64, err error) {
+	err = db.Pool.QueryRow(ctx, `SELECT count(*), coalesce(max(id), 0) FROM rpm_metadata WHERE repo_name = $1`, repoName).Scan(&count, &maxID)
+	return count, maxID, err
+}
+
 type RPMMetadataRow struct {
 	RepoName      string
 	FilePath      string

internal/database/virtuals.go

@@ -8,12 +8,12 @@ import (
 
 func (db *DB) GetVirtual(ctx context.Context, name string) (*models.Virtual, error) {
 	row := db.Pool.QueryRow(ctx, `
-		SELECT name, package_type, description, members, managed_by, created_at, updated_at
+		SELECT name, package_type, description, members, managed_by, parallel_fetch, created_at, updated_at
 		FROM virtuals WHERE name = $1
 	`, name)
 
 	var v models.Virtual
-	err := row.Scan(&v.Name, &v.PackageType, &v.Description, &v.Members, &v.ManagedBy, &v.CreatedAt, &v.UpdatedAt)
+	err := row.Scan(&v.Name, &v.PackageType, &v.Description, &v.Members, &v.ManagedBy, &v.ParallelFetch, &v.CreatedAt, &v.UpdatedAt)
 	if err != nil {
 		return nil, err
 	}
@@ -22,7 +22,7 @@ func (db *DB) GetVirtual(ctx context.Context, name string) (*models.Virtual, err
 
 func (db *DB) ListVirtuals(ctx context.Context) ([]models.Virtual, error) {
 	rows, err := db.Pool.Query(ctx, `
-		SELECT name, package_type, description, members, managed_by, created_at, updated_at
+		SELECT name, package_type, description, members, managed_by, parallel_fetch, created_at, updated_at
 		FROM virtuals ORDER BY name
 	`)
 	if err != nil {
@@ -33,7 +33,7 @@ func (db *DB) ListVirtuals(ctx context.Context) ([]models.Virtual, error) {
 	var virtuals []models.Virtual
 	for rows.Next() {
 		var v models.Virtual
-		if err := rows.Scan(&v.Name, &v.PackageType, &v.Description, &v.Members, &v.ManagedBy, &v.CreatedAt, &v.UpdatedAt); err != nil {
+		if err := rows.Scan(&v.Name, &v.PackageType, &v.Description, &v.Members, &v.ManagedBy, &v.ParallelFetch, &v.CreatedAt, &v.UpdatedAt); err != nil {
 			return nil, err
 		}
 		virtuals = append(virtuals, v)
@@ -43,18 +43,18 @@ func (db *DB) ListVirtuals(ctx context.Context) ([]models.Virtual, error) {
 
 func (db *DB) CreateVirtual(ctx context.Context, v *models.Virtual) error {
 	_, err := db.Pool.Exec(ctx, `
-		INSERT INTO virtuals (name, package_type, description, members, managed_by)
+		INSERT INTO virtuals (name, package_type, description, members, managed_by, parallel_fetch)
-		VALUES ($1, $2, $3, $4, $5)
+		VALUES ($1, $2, $3, $4, $5, $6)
-	`, v.Name, v.PackageType, v.Description, v.Members, v.ManagedBy)
+	`, v.Name, v.PackageType, v.Description, v.Members, v.ManagedBy, v.ParallelFetch)
 	return err
 }
 
 func (db *DB) UpdateVirtual(ctx context.Context, v *models.Virtual) error {
 	_, err := db.Pool.Exec(ctx, `
 		UPDATE virtuals SET
-			package_type=$2, description=$3, members=$4, managed_by=$5, updated_at=NOW()
+			package_type=$2, description=$3, members=$4, managed_by=$5, parallel_fetch=$6, updated_at=NOW()
 		WHERE name=$1
-	`, v.Name, v.PackageType, v.Description, v.Members, v.ManagedBy)
+	`, v.Name, v.PackageType, v.Description, v.Members, v.ManagedBy, v.ParallelFetch)
 	return err
 }
 

internal/provider/alpine/syncer.go

@@ -27,7 +27,7 @@ const (
 type SyncStore interface {
 	provider.RemoteMetadataStore
 	ListGitHubAlpineRemotes(ctx context.Context) ([]models.Remote, error)
-	ClaimGitHubAlpineSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (claimed bool, etag string, err error)
+	ClaimGitHubAlpineSyncLease(ctx context.Context, remoteName, owner string, freshness, lease, interval, phase time.Duration) (claimed bool, etag string, err error)
 	ReleaseGitHubAlpineSyncLease(ctx context.Context, remoteName, owner string, res provider.SyncResult) error
 }
 
@@ -129,7 +129,8 @@ func (s *Syncer) Run(ctx context.Context) {
 }
 
 // schedule enqueues a periodic check for every github_alpine remote. The DB lease
-// enforces the per-remote mutable_ttl cadence and cross-replica coordination.
+// enforces the per-remote sync_interval (else mutable_ttl) cadence and
+// cross-replica coordination.
 func (s *Syncer) schedule(ctx context.Context) {
 	remotes, err := s.store.ListGitHubAlpineRemotes(ctx)
 	if err != nil {
@@ -142,8 +143,10 @@ func (s *Syncer) schedule(ctx context.Context) {
 }
 
 // EnqueuePrime queues an immediate background prime for a freshly created remote.
+// A sync_interval remote skips it and syncs in its first slot, so remotes
+// created together do not burst.
 func (s *Syncer) EnqueuePrime(remote models.Remote) {
-	if s == nil {
+	if s == nil || remote.SyncInterval > 0 {
 		return
 	}
 	s.enqueue(remote, true)
@@ -189,16 +192,16 @@ func (s *Syncer) process(ctx context.Context, job syncJob) {
 		s.mu.Unlock()
 	}()
 
-	ttl := time.Duration(job.remote.MutableTTL) * time.Second
+	ttl := job.remote.SyncPeriod(defaultSyncFreshness)
-	if ttl <= 0 {
-		ttl = defaultSyncFreshness
-	}
 	freshness := ttl
+	var interval, phase time.Duration
 	if job.prime {
 		freshness = 0
+	} else {
+		interval, phase = job.remote.SyncSchedule()
 	}
 
-	claimed, etag, err := s.store.ClaimGitHubAlpineSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration)
+	claimed, etag, err := s.store.ClaimGitHubAlpineSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration, interval, phase)
 	if err != nil {
 		slog.Error("github_alpine syncer: claim lease", "remote", job.remote.Name, "error", err)
 		return

internal/provider/alpine/syncer_test.go

@@ -28,6 +28,10 @@ type fakeSyncStore struct {
 	lastSynced map[string]time.Time
 	etags      map[string]string
 	retryAt    map[string]time.Time
+	freshness  time.Duration
+	interval   time.Duration
+	phase      time.Duration
+	attempts   int
 }
 
 func newFakeSyncStore() *fakeSyncStore {
@@ -47,9 +51,11 @@ func (f *fakeSyncStore) ListGitHubAlpineRemotes(_ context.Context) ([]models.Rem
 	return append([]models.Remote(nil), f.remotes...), nil
 }
 
-func (f *fakeSyncStore) ClaimGitHubAlpineSyncLease(_ context.Context, name, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (f *fakeSyncStore) ClaimGitHubAlpineSyncLease(_ context.Context, name, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
 	f.mu.Lock()
 	defer f.mu.Unlock()
+	f.freshness, f.interval, f.phase = freshness, interval, phase
+	f.attempts++
 	now := time.Now()
 	ls, hasLS := f.lastSynced[name]
 	exp, hasExp := f.leaseExp[name]
@@ -223,7 +229,7 @@ func TestSyncerLeasePreventsSecondReplica(t *testing.T) {
 	s := newSyncer(store, p, testSyncConfig())
 	remote := fx.remote()
 
-	claimed, _, err := store.ClaimGitHubAlpineSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration)
+	claimed, _, err := store.ClaimGitHubAlpineSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration, 0, 0)
 	if err != nil || !claimed {
 		t.Fatalf("replica-1 claim: claimed=%v err=%v", claimed, err)
 	}
@@ -363,3 +369,37 @@ func TestSyncerFailedAssetRetry(t *testing.T) {
 		})
 	}
 }
+
+// A periodic check claims with sync_interval as its freshness and passes the
+// remote's slot schedule; unset, it falls back to mutable_ttl with no schedule.
+func TestSyncerHonoursSyncInterval(t *testing.T) {
+	for _, interval := range []int{0, 7200} {
+		store := newFakeSyncStore()
+		s := newSyncer(store, newTestProvider(), testSyncConfig())
+		store.lastSynced["sched"] = time.Now()
+		remote := models.Remote{Name: "sched", MutableTTL: 90, SyncInterval: interval}
+		wantInterval, wantPhase := remote.SyncSchedule()
+		s.process(context.Background(), syncJob{remote: remote})
+		if want := remote.SyncPeriod(0); store.freshness != want || store.interval != wantInterval || store.phase != wantPhase {
+			t.Errorf("sync_interval=%d: claim freshness/interval/phase = %v/%v/%v, want %v/%v/%v",
+				interval, store.freshness, store.interval, store.phase, want, wantInterval, wantPhase)
+		}
+	}
+}
+
+// Create-time primes are skipped for a scheduled remote so remotes created
+// together spread out; a cold-cache prime still claims with no schedule.
+func TestSyncerScheduledPrime(t *testing.T) {
+	store := newFakeSyncStore()
+	s := newSyncer(store, newTestProvider(), testSyncConfig())
+	remote := models.Remote{Name: "fresh", SyncInterval: 3600}
+
+	s.EnqueuePrime(remote)
+	if len(s.jobs) != 0 {
+		t.Fatal("create-time prime enqueued for a sync_interval remote")
+	}
+	s.process(context.Background(), syncJob{remote: remote, prime: true})
+	if store.attempts != 1 || store.freshness != 0 || store.interval != 0 {
+		t.Fatalf("cold-cache prime: attempts=%d freshness=%v interval=%v, want 1/0/0", store.attempts, store.freshness, store.interval)
+	}
+}

internal/provider/deb/syncer.go

@@ -27,7 +27,7 @@ const (
 type SyncStore interface {
 	provider.RemoteMetadataStore
 	ListGitHubDebRemotes(ctx context.Context) ([]models.Remote, error)
-	ClaimGitHubDebSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (claimed bool, etag string, err error)
+	ClaimGitHubDebSyncLease(ctx context.Context, remoteName, owner string, freshness, lease, interval, phase time.Duration) (claimed bool, etag string, err error)
 	ReleaseGitHubDebSyncLease(ctx context.Context, remoteName, owner string, res provider.SyncResult) error
 }
 
@@ -129,7 +129,8 @@ func (s *Syncer) Run(ctx context.Context) {
 }
 
 // schedule enqueues a periodic check for every github_deb remote. The DB lease
-// enforces the per-remote mutable_ttl cadence and cross-replica coordination.
+// enforces the per-remote sync_interval (else mutable_ttl) cadence and
+// cross-replica coordination.
 func (s *Syncer) schedule(ctx context.Context) {
 	remotes, err := s.store.ListGitHubDebRemotes(ctx)
 	if err != nil {
@@ -142,8 +143,10 @@ func (s *Syncer) schedule(ctx context.Context) {
 }
 
 // EnqueuePrime queues an immediate background prime for a freshly created remote.
+// A sync_interval remote skips it and syncs in its first slot, so remotes
+// created together do not burst.
 func (s *Syncer) EnqueuePrime(remote models.Remote) {
-	if s == nil {
+	if s == nil || remote.SyncInterval > 0 {
 		return
 	}
 	s.enqueue(remote, true)
@@ -189,16 +192,16 @@ func (s *Syncer) process(ctx context.Context, job syncJob) {
 		s.mu.Unlock()
 	}()
 
-	ttl := time.Duration(job.remote.MutableTTL) * time.Second
+	ttl := job.remote.SyncPeriod(defaultSyncFreshness)
-	if ttl <= 0 {
-		ttl = defaultSyncFreshness
-	}
 	freshness := ttl
+	var interval, phase time.Duration
 	if job.prime {
 		freshness = 0
+	} else {
+		interval, phase = job.remote.SyncSchedule()
 	}
 
-	claimed, etag, err := s.store.ClaimGitHubDebSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration)
+	claimed, etag, err := s.store.ClaimGitHubDebSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration, interval, phase)
 	if err != nil {
 		slog.Error("github_deb syncer: claim lease", "remote", job.remote.Name, "error", err)
 		return

internal/provider/deb/syncer_test.go

@@ -28,6 +28,10 @@ type fakeSyncStore struct {
 	lastSynced map[string]time.Time
 	etags      map[string]string
 	retryAt    map[string]time.Time
+	freshness  time.Duration
+	interval   time.Duration
+	phase      time.Duration
+	attempts   int
 }
 
 func newFakeSyncStore() *fakeSyncStore {
@@ -47,9 +51,11 @@ func (f *fakeSyncStore) ListGitHubDebRemotes(_ context.Context) ([]models.Remote
 	return append([]models.Remote(nil), f.remotes...), nil
 }
 
-func (f *fakeSyncStore) ClaimGitHubDebSyncLease(_ context.Context, name, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (f *fakeSyncStore) ClaimGitHubDebSyncLease(_ context.Context, name, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
 	f.mu.Lock()
 	defer f.mu.Unlock()
+	f.freshness, f.interval, f.phase = freshness, interval, phase
+	f.attempts++
 	now := time.Now()
 	ls, hasLS := f.lastSynced[name]
 	exp, hasExp := f.leaseExp[name]
@@ -223,7 +229,7 @@ func TestSyncerLeasePreventsSecondReplica(t *testing.T) {
 	s := newSyncer(store, p, testSyncConfig())
 	remote := fx.remote()
 
-	claimed, _, err := store.ClaimGitHubDebSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration)
+	claimed, _, err := store.ClaimGitHubDebSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration, 0, 0)
 	if err != nil || !claimed {
 		t.Fatalf("replica-1 claim: claimed=%v err=%v", claimed, err)
 	}
@@ -363,3 +369,37 @@ func TestSyncerFailedAssetRetry(t *testing.T) {
 		})
 	}
 }
+
+// A periodic check claims with sync_interval as its freshness and passes the
+// remote's slot schedule; unset, it falls back to mutable_ttl with no schedule.
+func TestSyncerHonoursSyncInterval(t *testing.T) {
+	for _, interval := range []int{0, 7200} {
+		store := newFakeSyncStore()
+		s := newSyncer(store, newTestProvider(), testSyncConfig())
+		store.lastSynced["sched"] = time.Now()
+		remote := models.Remote{Name: "sched", MutableTTL: 90, SyncInterval: interval}
+		wantInterval, wantPhase := remote.SyncSchedule()
+		s.process(context.Background(), syncJob{remote: remote})
+		if want := remote.SyncPeriod(0); store.freshness != want || store.interval != wantInterval || store.phase != wantPhase {
+			t.Errorf("sync_interval=%d: claim freshness/interval/phase = %v/%v/%v, want %v/%v/%v",
+				interval, store.freshness, store.interval, store.phase, want, wantInterval, wantPhase)
+		}
+	}
+}
+
+// Create-time primes are skipped for a scheduled remote so remotes created
+// together spread out; a cold-cache prime still claims with no schedule.
+func TestSyncerScheduledPrime(t *testing.T) {
+	store := newFakeSyncStore()
+	s := newSyncer(store, newTestProvider(), testSyncConfig())
+	remote := models.Remote{Name: "fresh", SyncInterval: 3600}
+
+	s.EnqueuePrime(remote)
+	if len(s.jobs) != 0 {
+		t.Fatal("create-time prime enqueued for a sync_interval remote")
+	}
+	s.process(context.Background(), syncJob{remote: remote, prime: true})
+	if store.attempts != 1 || store.freshness != 0 || store.interval != 0 {
+		t.Fatalf("cold-cache prime: attempts=%d freshness=%v interval=%v, want 1/0/0", store.attempts, store.freshness, store.interval)
+	}
+}

internal/provider/rpm/github.go

@@ -15,9 +15,11 @@ import (
 	"regexp"
 	"strings"
 	"sync"
+	"sync/atomic"
 	"time"
 
 	rpmlib "github.com/cavaliergopher/rpm"
+	"golang.org/x/sync/singleflight"
 	"golang.org/x/time/rate"
 
 	"git.unkin.net/unkin/artifactapi/internal/githubauth"
@@ -52,6 +54,9 @@ const (
 	// retryable 503. Kept short so a client never hangs on a rate-limited derive
 	// of a large repo; small repos usually prime within this window.
 	defaultColdWait = 8 * time.Second
+	// repodataIdleTTL evicts cached indexes of remotes deleted or switched off.
+	repodataIdleTTL    = time.Hour
+	repodataSweepEvery = time.Minute
 )
 
 // GitHubProvider is a metadata-only remote: it scans a GitHub repo's releases
@@ -84,6 +89,26 @@ type GitHubProvider struct {
 	mu       sync.Mutex
 	scanning map[string]bool
 	lastScan map[string]time.Time
+
+	// repodata caches generated indexes for cache_repodata remotes, keyed by remote name.
+	repodataMu    sync.Mutex
+	repodataCache map[string]cachedRepodata
+	repodataGroup singleflight.Group
+	repodataSeq   atomic.Int64
+	repodataSwept time.Time
+}
+
+// rpmFingerprinter is a cheap change detector over a remote's metadata rows.
+// Rows are insert-or-delete only, so (count, max id) changes whenever they do.
+type rpmFingerprinter interface {
+	RPMMetadataFingerprint(ctx context.Context, repoName string) (count, maxID int64, err error)
+}
+
+type cachedRepodata struct {
+	count, maxID int64
+	seq          int64 // build order, so a slow older build never replaces a newer one
+	used         time.Time
+	*repodata
 }
 
 func newGitHubProvider() *GitHubProvider {
@@ -97,6 +122,7 @@ func newGitHubProvider() *GitHubProvider {
 		coldWait:      defaultColdWait,
 		scanning:      map[string]bool{},
 		lastScan:      map[string]time.Time{},
+		repodataCache: map[string]cachedRepodata{},
 	}
 }
 
@@ -170,6 +196,11 @@ func (p *GitHubProvider) ServeRemote(w http.ResponseWriter, r *http.Request, rem
 		}
 
 		tail := strings.TrimPrefix(path, "repodata/")
+		if fp, ok := store.(rpmFingerprinter); ok && remote.CacheRepodata {
+			p.serveCachedRepodata(w, sr, store, fp, remote.Name, tail)
+			return true
+		}
+		p.dropRepodata(remote.Name)
 		lp := &Provider{}
 		switch {
 		case tail == "repomd.xml":
@@ -239,7 +270,104 @@ func (p *GitHubProvider) ensurePrimed(ctx context.Context, remote models.Remote,
 	return false
 }
 
+// serveCachedRepodata serves a file from the generated index, rebuilding it
+// only when the rows' fingerprint moved. The fingerprint is read from the shared
+// database on every request, so each replica rebuilds as soon as any replica's
+// scan changes rows and repomd never outlives the files it references.
+func (p *GitHubProvider) serveCachedRepodata(w http.ResponseWriter, r *http.Request, store provider.RPMMetadataReader, fp rpmFingerprinter, name, tail string) {
+	var pick func(*repodata) []byte
+	contentType := "application/gzip"
+	switch {
+	case tail == "repomd.xml":
+		pick, contentType = func(rd *repodata) []byte { return rd.repomd }, "application/xml"
+	case strings.HasSuffix(tail, "-primary.xml.gz"):
+		pick = func(rd *repodata) []byte { return rd.primary }
+	case strings.HasSuffix(tail, "-filelists.xml.gz"):
+		pick = func(rd *repodata) []byte { return rd.filelists }
+	case strings.HasSuffix(tail, "-other.xml.gz"):
+		pick = func(rd *repodata) []byte { return rd.other }
+	default:
+		http.Error(w, "not found", http.StatusNotFound)
+		return
+	}
+	rd, err := p.cachedRepodata(r.Context(), store, fp, name)
+	if err != nil {
+		writeMetadataError(w, name, err)
+		return
+	}
+	w.Header().Set("Content-Type", contentType)
+	w.WriteHeader(http.StatusOK)
+	_, _ = w.Write(pick(rd))
+}
+
+// cachedRepodata returns the index for the current fingerprint, building it at
+// most once per fingerprint. ctx is ServeRemote's detached serve context, so a
+// leader's client disconnect cannot fail the build for coalesced waiters.
+func (p *GitHubProvider) cachedRepodata(ctx context.Context, store provider.RPMMetadataReader, fp rpmFingerprinter, name string) (*repodata, error) {
+	seq := p.repodataSeq.Add(1)
+	count, maxID, err := fp.RPMMetadataFingerprint(ctx, name)
+	if err != nil {
+		return nil, err
+	}
+	now := time.Now()
+	p.repodataMu.Lock()
+	p.sweepRepodataLocked(now)
+	c, ok := p.repodataCache[name]
+	if ok {
+		c.used = now
+		p.repodataCache[name] = c
+	}
+	p.repodataMu.Unlock()
+	if ok && c.count == count && c.maxID == maxID {
+		return c.repodata, nil
+	}
+
+	v, err, _ := p.repodataGroup.Do(fmt.Sprintf("%s\x00%d\x00%d", name, count, maxID), func() (any, error) {
+		metas, err := store.ListRPMMetadataEntries(ctx, name)
+		if err != nil {
+			return nil, err
+		}
+		// Rows read after the fingerprint can only be newer than it, so the
+		// worst case is one extra rebuild on the next request.
+		rd := buildRepodata(metas)
+		p.repodataMu.Lock()
+		if cur, ok := p.repodataCache[name]; !ok || cur.seq < seq {
+			p.repodataCache[name] = cachedRepodata{count: count, maxID: maxID, seq: seq, used: time.Now(), repodata: rd}
+		}
+		p.repodataMu.Unlock()
+		return rd, nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	return v.(*repodata), nil
+}
+
+func (p *GitHubProvider) dropRepodata(name string) {
+	p.repodataMu.Lock()
+	delete(p.repodataCache, name)
+	p.sweepRepodataLocked(time.Now())
+	p.repodataMu.Unlock()
+}
+
+// sweepRepodataLocked evicts idle entries, at most once per repodataSweepEvery.
+func (p *GitHubProvider) sweepRepodataLocked(now time.Time) {
+	if now.Sub(p.repodataSwept) < repodataSweepEvery {
+		return
+	}
+	p.repodataSwept = now
+	for name, c := range p.repodataCache {
+		if now.Sub(c.used) > repodataIdleTTL {
+			delete(p.repodataCache, name)
+		}
+	}
+}
+
 func (p *GitHubProvider) cacheEmpty(ctx context.Context, store provider.RemoteMetadataStore, name string) bool {
+	if fp, ok := store.(rpmFingerprinter); ok {
+		count, _, err := fp.RPMMetadataFingerprint(ctx, name)
+		return err == nil && count == 0
+	}
 	rows, err := store.ListRPMMetadataEntries(ctx, name)
 	if err != nil {
 		// Treat a failed read as "not empty" so a transient DB error becomes a

internal/provider/rpm/github_repodata_cache_test.go

@@ -0,0 +1,365 @@
+package rpm
+
+import (
+	"context"
+	"errors"
+	"net/http"
+	"net/http/httptest"
+	"runtime"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+
+	"git.unkin.net/unkin/artifactapi/internal/provider"
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+// fpStore adds a fingerprint and a full-read counter to fakeStore.
+type fpStore struct {
+	*fakeStore
+	lists, fps     int
+	seq            int64
+	fpErr, listErr error
+	gate           chan struct{} // when set, full reads block until it is closed
+}
+
+func (s *fpStore) ListRPMMetadataEntries(ctx context.Context, name string) ([]provider.RPMMetadata, error) {
+	s.mu.Lock()
+	s.lists++
+	gate, err := s.gate, s.listErr
+	s.mu.Unlock()
+	if gate != nil {
+		<-gate
+	}
+	if err != nil {
+		return nil, err
+	}
+	return s.fakeStore.ListRPMMetadataEntries(ctx, name)
+}
+
+func (s *fpStore) InsertRPMMetadata(ctx context.Context, m *provider.RPMMetadata) error {
+	s.mu.Lock()
+	_, exists := s.rows[m.FilePath]
+	s.mu.Unlock()
+	if err := s.fakeStore.InsertRPMMetadata(ctx, m); err != nil || exists {
+		return err // ON CONFLICT DO NOTHING leaves max(id) unchanged
+	}
+	s.mu.Lock()
+	s.seq++
+	s.mu.Unlock()
+	return nil
+}
+
+func (s *fpStore) RPMMetadataFingerprint(_ context.Context, _ string) (int64, int64, error) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.fps++
+	return int64(len(s.rows)), s.seq, s.fpErr
+}
+
+func (s *fpStore) fpCount() int {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return s.fps
+}
+
+func (s *fpStore) listCount() int {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return s.lists
+}
+
+func newCacheFixture(t *testing.T, cache bool) (*GitHubProvider, *fpStore, models.Remote) {
+	t.Helper()
+	remote := models.Remote{Name: "cached-rpm", PackageType: models.PackageGitHubRPM, MutableTTL: 3600, CacheRepodata: cache}
+	p := newGitHubProvider()
+	p.lastScan[remote.Name] = time.Now()
+	store := &fpStore{fakeStore: newFakeStore()}
+	addPkg(t, store, "alpha")
+	return p, store, remote
+}
+
+func addPkg(t *testing.T, store *fpStore, name string) {
+	t.Helper()
+	if err := store.InsertRPMMetadata(context.Background(), &provider.RPMMetadata{
+		RepoName: "cached-rpm", FilePath: name + "-1-1.noarch.rpm", ContentHash: "sha256:" + name,
+		Name: name, Version: "1", Release: "1", Arch: "noarch", Group: "Unspecified",
+		Files: []provider.RPMFile{{Path: "/usr/bin/" + name}},
+	}); err != nil {
+		t.Fatal(err)
+	}
+}
+
+func serveReq(t *testing.T, p *GitHubProvider, remote models.Remote, store *fpStore, req *http.Request, path string) *httptest.ResponseRecorder {
+	t.Helper()
+	rec := httptest.NewRecorder()
+	if !p.ServeRemote(rec, req, remote, path, "https://proxy.example", store) {
+		t.Errorf("%s not handled", path)
+	}
+	return rec
+}
+
+func serve(t *testing.T, p *GitHubProvider, remote models.Remote, store *fpStore, path string) []byte {
+	t.Helper()
+	rec := serveReq(t, p, remote, store, httptest.NewRequest(http.MethodGet, "/x", nil), path)
+	if rec.Code != http.StatusOK {
+		t.Fatalf("%s: code %d: %s", path, rec.Code, rec.Body.String())
+	}
+	return rec.Body.Bytes()
+}
+
+var repodataPaths = []string{
+	"repodata/repomd.xml",
+	"repodata/x-primary.xml.gz",
+	"repodata/x-filelists.xml.gz",
+	"repodata/x-other.xml.gz",
+}
+
+func TestCachedRepodataHitSkipsRegeneration(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+
+	repomd := serve(t, p, remote, store, "repodata/repomd.xml")
+	primary := serve(t, p, remote, store, "repodata/x-primary.xml.gz")
+	if n := store.listCount(); n != 1 {
+		t.Fatalf("full metadata reads = %d, want 1", n)
+	}
+	if !strings.Contains(string(repomd), sha256Hex(primary)+"-primary.xml.gz") {
+		t.Fatalf("cached repomd does not reference cached primary:\n%s", repomd)
+	}
+}
+
+func TestCachedRepodataRebuildsOnDataChange(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	before := serve(t, p, remote, store, "repodata/repomd.xml")
+
+	addPkg(t, store, "beta")
+	after := serve(t, p, remote, store, "repodata/repomd.xml")
+	primary := serve(t, p, remote, store, "repodata/x-primary.xml.gz")
+
+	if string(before) == string(after) {
+		t.Fatal("repomd unchanged after metadata change")
+	}
+	if !strings.Contains(string(after), sha256Hex(primary)+"-primary.xml.gz") {
+		t.Fatal("rebuilt repomd does not reference rebuilt primary")
+	}
+	if !strings.Contains(gunzip(t, primary), "<name>beta</name>") {
+		t.Fatal("rebuilt primary missing new package")
+	}
+	if n := store.listCount(); n != 2 {
+		t.Fatalf("full metadata reads = %d, want 2", n)
+	}
+}
+
+func TestUncachedRepodataMatchesCachedAndReadsEveryTime(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	cached := map[string]string{}
+	for _, path := range repodataPaths {
+		cached[path] = string(serve(t, p, remote, store, path))
+	}
+
+	remote.CacheRepodata = false
+	reads := store.listCount()
+	for _, path := range repodataPaths {
+		if got := string(serve(t, p, remote, store, path)); got != cached[path] {
+			t.Fatalf("%s differs between cached and uncached", path)
+		}
+	}
+	if n := store.listCount() - reads; n != len(repodataPaths) {
+		t.Fatalf("uncached full reads = %d, want %d", n, len(repodataPaths))
+	}
+	if _, ok := p.repodataCache[remote.Name]; ok {
+		t.Fatal("disabling cache_repodata should drop the cached index")
+	}
+}
+
+func TestCachedRepodataRebuildsOnSameCountReinsert(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	addPkg(t, store, "alpha") // conflicting insert: fingerprint must not move
+	serve(t, p, remote, store, "repodata/repomd.xml")
+	serve(t, p, remote, store, "repodata/repomd.xml")
+	if n := store.listCount(); n != 1 {
+		t.Fatalf("full reads after conflicting insert = %d, want 1", n)
+	}
+
+	if err := store.DeleteRPMMetadata(context.Background(), remote.Name, "alpha-1-1.noarch.rpm"); err != nil {
+		t.Fatal(err)
+	}
+	addPkg(t, store, "alpha")
+	serve(t, p, remote, store, "repodata/repomd.xml")
+	if n := store.listCount(); n != 2 {
+		t.Fatalf("full reads after same-count reinsert = %d, want 2", n)
+	}
+}
+
+func TestCachedRepodataEvictsIdleRemotes(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	serve(t, p, remote, store, "repodata/repomd.xml")
+	p.repodataCache["gone"] = cachedRepodata{used: time.Now().Add(-2 * repodataIdleTTL), repodata: &repodata{}}
+	p.repodataSwept = time.Time{}
+
+	serve(t, p, remote, store, "repodata/repomd.xml")
+	if _, ok := p.repodataCache["gone"]; ok {
+		t.Fatal("idle remote's index not evicted")
+	}
+	if _, ok := p.repodataCache[remote.Name]; !ok {
+		t.Fatal("active remote's index evicted")
+	}
+}
+
+func waitFor(t *testing.T, cond func() bool) {
+	t.Helper()
+	deadline := time.Now().Add(10 * time.Second)
+	for !cond() {
+		if time.Now().After(deadline) {
+			t.Fatal("condition not reached")
+		}
+		time.Sleep(time.Millisecond)
+	}
+}
+
+// sfWaiters counts goroutines parked on an in-flight singleflight call.
+func sfWaiters() int {
+	buf := make([]byte, 1<<20)
+	for {
+		n := runtime.Stack(buf, true)
+		if n < len(buf) {
+			buf = buf[:n]
+			break
+		}
+		buf = make([]byte, 2*len(buf))
+	}
+	waiters := 0
+	for _, g := range strings.Split(string(buf), "\n\n") {
+		if strings.Contains(g, "sync.(*WaitGroup).Wait") && strings.Contains(g, "singleflight.(*Group).Do(") {
+			waiters++
+		}
+	}
+	return waiters
+}
+
+func TestCachedRepodataCoalescesConcurrentBuilds(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	gate := make(chan struct{})
+	store.gate = gate
+	const n = 8
+	var wg sync.WaitGroup
+	codes := make([]int, n)
+	for i := range n {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			codes[i] = serveReq(t, p, remote, store, httptest.NewRequest(http.MethodGet, "/x", nil), repodataPaths[i%len(repodataPaths)]).Code
+		}()
+	}
+	waitFor(t, func() bool { return store.listCount() == 1 && sfWaiters() == n-1 })
+	close(gate)
+	wg.Wait()
+	for i, c := range codes {
+		if c != http.StatusOK {
+			t.Errorf("request %d: code %d", i, c)
+		}
+	}
+	if got := store.listCount(); got != 1 {
+		t.Fatalf("full metadata reads = %d, want 1", got)
+	}
+}
+
+func TestCachedRepodataLeaderCancelDoesNotFailWaiters(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	gate := make(chan struct{})
+	store.gate = gate
+	leaderCtx, cancelLeader := context.WithCancel(context.Background())
+	results := make(chan int, 2)
+	go func() {
+		req := httptest.NewRequest(http.MethodGet, "/x", nil).WithContext(leaderCtx)
+		results <- serveReq(t, p, remote, store, req, "repodata/repomd.xml").Code
+	}()
+	waitFor(t, func() bool { return store.listCount() == 1 })
+	go func() {
+		results <- serveReq(t, p, remote, store, httptest.NewRequest(http.MethodGet, "/x", nil), "repodata/repomd.xml").Code
+	}()
+	waitFor(t, func() bool { return sfWaiters() == 1 })
+	cancelLeader()
+	close(gate)
+	for range 2 {
+		if c := <-results; c != http.StatusOK {
+			t.Fatalf("code %d after leader cancel, want 200", c)
+		}
+	}
+	if got := store.listCount(); got != 1 {
+		t.Fatalf("full metadata reads = %d, want 1", got)
+	}
+}
+
+func TestCachedRepodataOlderBuildDoesNotOverwriteNewer(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	newer := &repodata{}
+	p.repodataCache[remote.Name] = cachedRepodata{count: 99, maxID: 99, seq: 1 << 40, used: time.Now(), repodata: newer}
+	serve(t, p, remote, store, "repodata/repomd.xml")
+	if p.repodataCache[remote.Name].repodata != newer {
+		t.Fatal("older build replaced a newer cache entry")
+	}
+}
+
+func TestCachedRepodataErrors(t *testing.T) {
+	secret := errors.New("pq: password authentication failed for user secret")
+	for _, tc := range []struct {
+		name           string
+		fpErr, listErr error
+		code           int
+	}{
+		{"fingerprint", secret, nil, http.StatusInternalServerError},
+		{"list", nil, secret, http.StatusInternalServerError},
+		{"fingerprint deadline", context.DeadlineExceeded, nil, http.StatusServiceUnavailable},
+		{"list canceled", nil, context.Canceled, http.StatusServiceUnavailable},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			p, store, remote := newCacheFixture(t, true)
+			store.fpErr, store.listErr = tc.fpErr, tc.listErr
+			rec := serveReq(t, p, remote, store, httptest.NewRequest(http.MethodGet, "/x", nil), "repodata/repomd.xml")
+			if rec.Code != tc.code {
+				t.Fatalf("code %d, want %d", rec.Code, tc.code)
+			}
+			if strings.Contains(rec.Body.String(), "secret") {
+				t.Fatalf("DB error leaked: %q", rec.Body.String())
+			}
+			if retry := rec.Header().Get("Retry-After"); (tc.code == http.StatusServiceUnavailable) != (retry != "") {
+				t.Fatalf("Retry-After = %q for %d", retry, tc.code)
+			}
+			if _, ok := p.repodataCache[remote.Name]; ok {
+				t.Fatal("failed build was cached")
+			}
+		})
+	}
+}
+
+func TestCachedRepodataUnknownTail404sWithoutDB(t *testing.T) {
+	p, store, remote := newCacheFixture(t, true)
+	rec := serveReq(t, p, remote, store, httptest.NewRequest(http.MethodGet, "/x", nil), "repodata/bogus.txt")
+	if rec.Code != http.StatusNotFound {
+		t.Fatalf("code %d, want 404", rec.Code)
+	}
+	if store.fpCount() != 0 || store.listCount() != 0 {
+		t.Fatalf("unknown tail hit the DB: fps=%d lists=%d", store.fpCount(), store.listCount())
+	}
+}
+
+func TestCacheEmptyUsesFingerprint(t *testing.T) {
+	p := newGitHubProvider()
+	store := &fpStore{fakeStore: newFakeStore()}
+	if !p.cacheEmpty(context.Background(), store, "cached-rpm") {
+		t.Fatal("empty store not reported empty")
+	}
+	addPkg(t, store, "alpha")
+	if p.cacheEmpty(context.Background(), store, "cached-rpm") {
+		t.Fatal("populated store reported empty")
+	}
+	store.fpErr = errors.New("db down")
+	if p.cacheEmpty(context.Background(), store, "cached-rpm") {
+		t.Fatal("fingerprint error must not report empty")
+	}
+	if store.listCount() != 0 {
+		t.Fatalf("cacheEmpty did full reads: %d", store.listCount())
+	}
+}

internal/provider/rpm/rpm.go

@@ -250,36 +250,52 @@ func (p *Provider) GenerateLocalIndex(ctx context.Context, files provider.FileSt
 func readMetadataEntries(w http.ResponseWriter, r *http.Request, reader provider.RPMMetadataReader, repoName string) ([]provider.RPMMetadata, bool) {
 	metas, err := reader.ListRPMMetadataEntries(r.Context(), repoName)
 	if err != nil {
-		if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
+		writeMetadataError(w, repoName, err)
-			slog.Warn("rpm: metadata read canceled", "repo", repoName, "error", err)
-			http.Error(w, "metadata read canceled", http.StatusServiceUnavailable)
-			return nil, false
-		}
-		http.Error(w, err.Error(), http.StatusInternalServerError)
 		return nil, false
 	}
 	return metas, true
 }
 
+// writeMetadataError maps a metadata read failure to a response without leaking
+// DB error text: a canceled/expired context is a retryable 503, anything else a 500.
+func writeMetadataError(w http.ResponseWriter, repoName string, err error) {
+	if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
+		slog.Warn("rpm: metadata read canceled", "repo", repoName, "error", err)
+		w.Header().Set("Retry-After", "5")
+		http.Error(w, "metadata read canceled", http.StatusServiceUnavailable)
+		return
+	}
+	slog.Error("rpm: metadata read failed", "repo", repoName, "error", err)
+	http.Error(w, "metadata read failed", http.StatusInternalServerError)
+}
+
 func (p *Provider) serveRepomd(w http.ResponseWriter, r *http.Request, reader provider.RPMMetadataReader, repoName string) {
 	metas, ok := readMetadataEntries(w, r, reader, repoName)
 	if !ok {
 		return
 	}
 
-	primary := generatePrimaryXMLGZ(metas)
-	filelists := generateFilelistsXMLGZ(metas)
-	other := generateOtherXMLGZ(metas)
-
-	primaryHash := sha256Hex(primary)
-	filelistsHash := sha256Hex(filelists)
-	otherHash := sha256Hex(other)
-
-	repomd := generateRepomd(repomdRevision(metas), primaryHash, len(primary), filelistsHash, len(filelists), otherHash, len(other))
-
 	w.Header().Set("Content-Type", "application/xml")
 	w.WriteHeader(http.StatusOK)
-	w.Write(repomd)
+	_, _ = w.Write(buildRepodata(metas).repomd)
+}
+
+// repodata is a complete generated index: repomd.xml plus the three gz files it references.
+type repodata struct {
+	repomd, primary, filelists, other []byte
+}
+
+func buildRepodata(metas []provider.RPMMetadata) *repodata {
+	rd := &repodata{
+		primary:   generatePrimaryXMLGZ(metas),
+		filelists: generateFilelistsXMLGZ(metas),
+		other:     generateOtherXMLGZ(metas),
+	}
+	rd.repomd = generateRepomd(repomdRevision(metas),
+		sha256Hex(rd.primary), len(rd.primary),
+		sha256Hex(rd.filelists), len(rd.filelists),
+		sha256Hex(rd.other), len(rd.other))
+	return rd
 }
 
 func (p *Provider) servePrimary(w http.ResponseWriter, r *http.Request, reader provider.RPMMetadataReader, repoName string) {

internal/provider/rpm/syncer.go

@@ -21,7 +21,7 @@ const (
 	// scan never loses its lease, while a crashed replica's lease still expires.
 	syncLeaseDuration = 15 * time.Minute
 	// defaultSyncFreshness is the periodic re-check interval used when a remote's
-	// mutable_ttl is unset.
+	// sync_interval and mutable_ttl are unset.
 	defaultSyncFreshness = 5 * time.Minute
 	// jobQueueDepth bounds the pending work queue; enqueues past it are dropped
 	// (a later poll re-enqueues), never blocking the caller.
@@ -34,7 +34,7 @@ const (
 type SyncStore interface {
 	provider.RemoteMetadataStore
 	ListGitHubRPMRemotes(ctx context.Context) ([]models.Remote, error)
-	ClaimGitHubSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (claimed bool, etag string, err error)
+	ClaimGitHubSyncLease(ctx context.Context, remoteName, owner string, freshness, lease, interval, phase time.Duration) (claimed bool, etag string, err error)
 	ReleaseGitHubSyncLease(ctx context.Context, remoteName, owner string, res provider.SyncResult) error
 }
 
@@ -43,7 +43,7 @@ type SyncConfig struct {
 	RatePerSec   float64       // global GitHub request rate (req/s)
 	Burst        int           // token-bucket burst
 	Workers      int           // concurrent scan workers
-	PollInterval time.Duration // base scheduler tick; per-remote cadence is mutable_ttl
+	PollInterval time.Duration // base scheduler tick; per-remote cadence is sync_interval or mutable_ttl
 }
 
 type syncJob struct {
@@ -139,9 +139,9 @@ func (s *Syncer) Run(ctx context.Context) {
 }
 
 // schedule enqueues a periodic check for every github_rpm remote. The DB lease
-// (claimed in the worker) enforces the per-remote mutable_ttl cadence and cross
+// (claimed in the worker) enforces the per-remote sync_interval (else
-// replica coordination, so enqueuing every tick is cheap: a not-yet-due remote
+// mutable_ttl) cadence and cross replica coordination, so enqueuing every tick
-// simply fails to claim and is skipped.
+// is cheap: a not-yet-due remote simply fails to claim and is skipped.
 func (s *Syncer) schedule(ctx context.Context) {
 	remotes, err := s.store.ListGitHubRPMRemotes(ctx)
 	if err != nil {
@@ -155,8 +155,10 @@ func (s *Syncer) schedule(ctx context.Context) {
 
 // EnqueuePrime queues an immediate background prime for a freshly created
 // remote so its metadata is derived without blocking the create call.
+// A sync_interval remote skips it and syncs in its first slot, so remotes
+// created together do not burst.
 func (s *Syncer) EnqueuePrime(remote models.Remote) {
-	if s == nil {
+	if s == nil || remote.SyncInterval > 0 {
 		return
 	}
 	s.enqueue(remote, true)
@@ -205,16 +207,16 @@ func (s *Syncer) process(ctx context.Context, job syncJob) {
 		s.mu.Unlock()
 	}()
 
-	ttl := time.Duration(job.remote.MutableTTL) * time.Second
+	ttl := job.remote.SyncPeriod(defaultSyncFreshness)
-	if ttl <= 0 {
-		ttl = defaultSyncFreshness
-	}
 	freshness := ttl
+	var interval, phase time.Duration
 	if job.prime {
 		freshness = 0 // prime ignores the recency gate but still respects a live lease
+	} else {
+		interval, phase = job.remote.SyncSchedule()
 	}
 
-	claimed, etag, err := s.store.ClaimGitHubSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration)
+	claimed, etag, err := s.store.ClaimGitHubSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration, interval, phase)
 	if err != nil {
 		slog.Error("github_rpm syncer: claim lease", "remote", job.remote.Name, "error", err)
 		return

internal/provider/rpm/syncer_test.go

@@ -29,6 +29,10 @@ type fakeSyncStore struct {
 	etags      map[string]string
 	retryAt    map[string]time.Time
 	lastResult provider.SyncResult
+	freshness  time.Duration
+	interval   time.Duration
+	phase      time.Duration
+	attempts   int
 }
 
 func newFakeSyncStore() *fakeSyncStore {
@@ -48,9 +52,11 @@ func (f *fakeSyncStore) ListGitHubRPMRemotes(_ context.Context) ([]models.Remote
 	return append([]models.Remote(nil), f.remotes...), nil
 }
 
-func (f *fakeSyncStore) ClaimGitHubSyncLease(_ context.Context, name, owner string, freshness, lease time.Duration) (bool, string, error) {
+func (f *fakeSyncStore) ClaimGitHubSyncLease(_ context.Context, name, owner string, freshness, lease, interval, phase time.Duration) (bool, string, error) {
 	f.mu.Lock()
 	defer f.mu.Unlock()
+	f.freshness, f.interval, f.phase = freshness, interval, phase
+	f.attempts++
 	now := time.Now()
 	ls, hasLS := f.lastSynced[name]
 	exp, hasExp := f.leaseExp[name]
@@ -230,7 +236,7 @@ func TestSyncerLeasePreventsSecondReplica(t *testing.T) {
 	remote := fx.remote()
 
 	// Replica 1 holds the lease.
-	claimed, _, err := store.ClaimGitHubSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration)
+	claimed, _, err := store.ClaimGitHubSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration, 0, 0)
 	if err != nil || !claimed {
 		t.Fatalf("replica-1 claim: claimed=%v err=%v", claimed, err)
 	}
@@ -470,3 +476,43 @@ func TestSyncerFailedAssetRetry(t *testing.T) {
 		})
 	}
 }
+
+// A periodic check claims with sync_interval as its freshness and passes the
+// remote's slot schedule; unset, it falls back to mutable_ttl with no schedule.
+func TestSyncerHonoursSyncInterval(t *testing.T) {
+	for _, interval := range []int{0, 7200} {
+		store := newFakeSyncStore()
+		s := newSyncer(store, newTestProvider(), testSyncConfig())
+		store.lastSynced["sched"] = time.Now()
+		remote := models.Remote{Name: "sched", MutableTTL: 90, SyncInterval: interval}
+		wantInterval, wantPhase := remote.SyncSchedule()
+		s.process(context.Background(), syncJob{remote: remote})
+		if want := remote.SyncPeriod(0); store.freshness != want || store.interval != wantInterval || store.phase != wantPhase {
+			t.Errorf("sync_interval=%d: claim freshness/interval/phase = %v/%v/%v, want %v/%v/%v",
+				interval, store.freshness, store.interval, store.phase, want, wantInterval, wantPhase)
+		}
+	}
+}
+
+// Create-time primes are skipped for a scheduled remote so remotes created
+// together spread out; a cold-cache prime still claims with no schedule.
+func TestSyncerScheduledPrime(t *testing.T) {
+	store := newFakeSyncStore()
+	s := newSyncer(store, newTestProvider(), testSyncConfig())
+	remote := models.Remote{Name: "fresh", SyncInterval: 3600}
+
+	s.EnqueuePrime(remote)
+	if len(s.jobs) != 0 {
+		t.Fatal("create-time prime enqueued for a sync_interval remote")
+	}
+	s.process(context.Background(), syncJob{remote: remote, prime: true})
+	if store.attempts != 1 || store.freshness != 0 || store.interval != 0 {
+		t.Fatalf("cold-cache prime: attempts=%d freshness=%v interval=%v, want 1/0/0", store.attempts, store.freshness, store.interval)
+	}
+}
+
+func TestSyncLeaseOutlivesScanTimeout(t *testing.T) {
+	if syncLeaseDuration <= defaultScanTimeout {
+		t.Fatalf("syncLeaseDuration %v must exceed defaultScanTimeout %v so scans stay serialised", syncLeaseDuration, defaultScanTimeout)
+	}
+}

internal/proxy/engine.go

@@ -325,12 +325,12 @@ func (e *Engine) Head(ctx context.Context, remote models.Remote, path string, pr
 		return &HeadResult{ContentType: info.ContentType, Size: info.Size, Source: "cache"}, nil
 	}
 
-	return e.headUpstream(ctx, remote, path, prov)
+	return e.headUpstream(ctx, remote, path, prov, class)
 }
 
 // headUpstream issues an upstream HEAD, load-balancing across the remote's base
 // URLs and failing over to the next mirror on a network error or 5xx.
-func (e *Engine) headUpstream(ctx context.Context, remote models.Remote, path string, prov provider.Provider) (*HeadResult, error) {
+func (e *Engine) headUpstream(ctx context.Context, remote models.Remote, path string, prov provider.Provider, class Classification) (*HeadResult, error) {
 	order := e.baseURLAttemptOrder(remote)
 	if len(order) == 0 {
 		return nil, &ProxyError{Status: http.StatusBadGateway, Message: "no upstream base_url configured"}
@@ -338,7 +338,7 @@ func (e *Engine) headUpstream(ctx context.Context, remote models.Remote, path st
 	var lastErr error
 	for i, url := range order {
 		ctr := e.beginAttempt(remote, url)
-		result, err := e.headUpstreamOnce(ctx, withBaseURL(remote, url), path, prov)
+		result, err := e.headUpstreamOnce(ctx, withBaseURL(remote, url), path, prov, class)
 		endAttempt(ctr)
 		if err == nil {
 			return result, nil
@@ -353,7 +353,7 @@ func (e *Engine) headUpstream(ctx context.Context, remote models.Remote, path st
 	return nil, lastErr
 }
 
-func (e *Engine) headUpstreamOnce(ctx context.Context, remote models.Remote, path string, prov provider.Provider) (*HeadResult, error) {
+func (e *Engine) headUpstreamOnce(ctx context.Context, remote models.Remote, path string, prov provider.Provider, class Classification) (*HeadResult, error) {
 	url := prov.UpstreamURL(remote, path)
 
 	authHeaders, err := prov.AuthHeaders(ctx, remote)
@@ -400,6 +400,11 @@ func (e *Engine) headUpstreamOnce(ctx context.Context, remote models.Remote, pat
 	if resp.StatusCode != http.StatusOK {
 		return nil, &ProxyError{Status: resp.StatusCode, Message: fmt.Sprintf("upstream returned %d", resp.StatusCode)}
 	}
+	if class == ClassImmutable {
+		if err := e.checkQuarantine(ctx, remote, path, resp.Header); err != nil {
+			return nil, err
+		}
+	}
 
 	contentType := prov.ContentType(path)
 	if ct := resp.Header.Get("Content-Type"); ct != "" {
@@ -527,6 +532,11 @@ func (e *Engine) fetchFromUpstreamOnce(ctx context.Context, remote models.Remote
 		}, nil
 	}
 
+	if err := e.checkQuarantine(ctx, remote, path, resp.Header); err != nil {
+		_ = resp.Body.Close()
+		return nil, err
+	}
+
 	// Immutable blobs are streamed through the content-addressable store
 	// (tempfile -> sha256 -> S3) so arbitrarily large artifacts never sit
 	// fully in memory. Immutable content is never rewritten in the proxy path.
@@ -663,6 +673,43 @@ func (e *Engine) checkUpstreamOnce(ctx context.Context, remote models.Remote, pa
 	return resp.StatusCode == http.StatusNotModified, nil
 }
 
+// checkQuarantine refuses an uncached immutable artifact published less than
+// quarantine_days ago. The publish time is the upstream Last-Modified, capped
+// at now, else the first time this path was seen.
+func (e *Engine) checkQuarantine(ctx context.Context, remote models.Remote, path string, upstream http.Header) error {
+	if !remote.QuarantineEnabled {
+		return nil
+	}
+	if a, err := e.db.GetArtifact(ctx, remote.Name, path); err == nil && a != nil {
+		return nil
+	}
+	days := remote.QuarantineDays
+	if days < 1 {
+		days = models.DefaultQuarantineDays
+	}
+	window := time.Duration(days) * 24 * time.Hour
+	now := time.Now()
+	published, err := http.ParseTime(upstream.Get("Last-Modified"))
+	if err != nil {
+		if published, err = e.cache.FirstSeen(ctx, remote.Name, path, window+24*time.Hour); err != nil {
+			slog.Warn("first-seen lookup failed", "remote", remote.Name, "path", path, "error", err)
+			return &ProxyError{Status: http.StatusServiceUnavailable, Message: "quarantine check unavailable", RetryAfter: 30}
+		}
+	}
+	if published.After(now) {
+		published = now
+	}
+	if until := published.Add(window); now.Before(until) {
+		return &ProxyError{
+			Status:    http.StatusForbidden,
+			Message:   fmt.Sprintf("quarantined: published %s, available from %s", published.UTC().Format(time.RFC3339), until.UTC().Format(time.RFC3339)),
+			Published: published.UTC(),
+			Available: until.UTC(),
+		}
+	}
+	return nil
+}
+
 // etagTTL keeps a mutable path's ETag well past its freshness key so an expired
 // path revalidates with a conditional HEAD, while idle paths still age out.
 func etagTTL(ttl time.Duration) time.Duration {
@@ -821,10 +868,21 @@ func fetchBearerToken(ctx context.Context, wwwAuth string, remote models.Remote)
 type ProxyError struct {
 	Status  int
 	Message string
+	// Published and Available are set only on a quarantine refusal.
+	Published time.Time
+	Available time.Time
+	// RetryAfter, in seconds, is sent as Retry-After when set.
+	RetryAfter int
 }
 
 func (e *ProxyError) Error() string { return e.Message }
 
+// Quarantined reports whether err is a quarantine refusal.
+func Quarantined(err error) bool {
+	var pe *ProxyError
+	return errors.As(err, &pe) && !pe.Published.IsZero()
+}
+
 type UpstreamError struct {
 	Err error
 }

internal/proxy/engine_test.go

@@ -227,7 +227,7 @@ func TestHeadUpstreamUsesRemoteTimeout(t *testing.T) {
 	r := models.Remote{Name: "slow-head", PackageType: models.PackageGeneric, BaseURL: slow.URL, UpstreamResponseHeaderTimeout: 1}
 
 	start := time.Now()
-	_, err := (&Engine{}).headUpstreamOnce(context.Background(), r, "blob.bin", prov(t, models.PackageGeneric))
+	_, err := (&Engine{}).headUpstreamOnce(context.Background(), r, "blob.bin", prov(t, models.PackageGeneric), ClassImmutable)
 	if err == nil || time.Since(start) > 2*time.Second {
 		t.Fatalf("HEAD ignored the remote's response header timeout: err=%v after %s", err, time.Since(start))
 	}

internal/proxy/quarantine_test.go

@@ -0,0 +1,186 @@
+package proxy
+
+import (
+	"context"
+	"errors"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+	"time"
+
+	"git.unkin.net/unkin/artifactapi/internal/cache"
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+	goredis "github.com/redis/go-redis/v9"
+)
+
+func must[T any](v T, err error) T {
+	if err != nil {
+		panic(err)
+	}
+	return v
+}
+
+func quarantineUpstream(t *testing.T) *httptest.Server {
+	t.Helper()
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/fresh.tgz":
+			w.Header().Set("Last-Modified", time.Now().Add(-time.Hour).UTC().Format(http.TimeFormat))
+		case "/old.tgz":
+			w.Header().Set("Last-Modified", time.Now().Add(-30*24*time.Hour).UTC().Format(http.TimeFormat))
+		case "/future.tgz":
+			w.Header().Set("Last-Modified", time.Now().Add(30*24*time.Hour).UTC().Format(http.TimeFormat))
+		case "/undated.tgz", "/pkg":
+		default:
+			http.NotFound(w, r)
+			return
+		}
+		_, _ = w.Write([]byte("body " + r.URL.Path))
+	}))
+	t.Cleanup(srv.Close)
+	return srv
+}
+
+func quarantineRemote(t *testing.T, name string, enabled bool) models.Remote {
+	r := models.Remote{Name: name, PackageType: models.PackageNPM, RepoType: models.RepoTypeRemote,
+		BaseURL: quarantineUpstream(t).URL, QuarantineEnabled: enabled, QuarantineDays: 3}
+	return seed(t, r)
+}
+
+func wantQuarantined(t *testing.T, err error) *ProxyError {
+	t.Helper()
+	var pe *ProxyError
+	if !errors.As(err, &pe) || pe.Status != http.StatusForbidden || !Quarantined(err) {
+		t.Fatalf("want quarantine 403, got %v", err)
+	}
+	return pe
+}
+
+func TestQuarantineBlocksFreshArtifact(t *testing.T) {
+	requireStack(t)
+	ctx := context.Background()
+	r := quarantineRemote(t, "q-fresh", true)
+	pe := wantQuarantined(t, func() error { _, err := testEngine.Fetch(ctx, r, "fresh.tgz", prov(t, models.PackageNPM)); return err }())
+	if d := time.Until(pe.Available); d < 70*time.Hour || d > 72*time.Hour {
+		t.Errorf("available in %s, want ~71h", d)
+	}
+	if a, _ := testDB.GetArtifact(ctx, r.Name, "fresh.tgz"); a != nil {
+		t.Error("quarantined artifact was cached")
+	}
+	_, err := testEngine.Head(ctx, r, "fresh.tgz", prov(t, models.PackageNPM))
+	_ = wantQuarantined(t, err)
+}
+
+func TestQuarantineServesOldArtifact(t *testing.T) {
+	requireStack(t)
+	r := quarantineRemote(t, "q-old", true)
+	res, err := testEngine.Fetch(context.Background(), r, "old.tgz", prov(t, models.PackageNPM))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got := readAll(t, res); got != "body /old.tgz" {
+		t.Errorf("body = %q", got)
+	}
+}
+
+func TestQuarantineFallsBackToFirstSeen(t *testing.T) {
+	requireStack(t)
+	ctx := context.Background()
+	r := quarantineRemote(t, "q-undated", true)
+	p := prov(t, models.PackageNPM)
+	_, err := testEngine.Fetch(ctx, r, "undated.tgz", p)
+	first := wantQuarantined(t, err).Published
+	if time.Since(first) > time.Minute {
+		t.Fatalf("first seen %s, want now", first)
+	}
+	_, err = testEngine.Fetch(ctx, r, "undated.tgz", p)
+	if again := wantQuarantined(t, err).Published; !again.Equal(first) {
+		t.Errorf("first seen moved: %s -> %s", first, again)
+	}
+
+	r.QuarantineDays = 1
+	c := goredis.NewClient(must(goredis.ParseURL(testRedis)))
+	defer func() { _ = c.Close() }()
+	if err := c.Set(ctx, "firstseen:q-undated:undated.tgz", time.Now().Add(-48*time.Hour).Unix(), time.Hour).Err(); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := testEngine.Fetch(ctx, r, "undated.tgz", p); err != nil {
+		t.Errorf("seen 48h ago with 1 day quarantine: %v", err)
+	}
+}
+
+func TestQuarantineSkipsMutable(t *testing.T) {
+	requireStack(t)
+	r := quarantineRemote(t, "q-mutable", true)
+	if _, err := testEngine.Fetch(context.Background(), r, "pkg", prov(t, models.PackageNPM)); err != nil {
+		t.Errorf("mutable index blocked: %v", err)
+	}
+}
+
+func TestQuarantineDisabled(t *testing.T) {
+	requireStack(t)
+	r := quarantineRemote(t, "q-off", false)
+	if _, err := testEngine.Fetch(context.Background(), r, "fresh.tgz", prov(t, models.PackageNPM)); err != nil {
+		t.Errorf("disabled quarantine blocked: %v", err)
+	}
+}
+
+func TestQuarantineKeepsServingCached(t *testing.T) {
+	requireStack(t)
+	ctx := context.Background()
+	r := quarantineRemote(t, "q-cached", false)
+	p := prov(t, models.PackageNPM)
+	if _, err := testEngine.Fetch(ctx, r, "fresh.tgz", p); err != nil {
+		t.Fatal(err)
+	}
+	r.QuarantineEnabled = true
+	expire(t, r.Name, "fresh.tgz")
+	if _, err := testEngine.Fetch(ctx, r, "fresh.tgz", p); err != nil {
+		t.Errorf("artifact cached before quarantine refused on refetch: %v", err)
+	}
+}
+
+func TestQuarantineCapsFutureLastModified(t *testing.T) {
+	requireStack(t)
+	r := quarantineRemote(t, "q-future", true)
+	_, err := testEngine.Fetch(context.Background(), r, "future.tgz", prov(t, models.PackageNPM))
+	pe := wantQuarantined(t, err)
+	if d := time.Until(pe.Available); d < 71*time.Hour || d > 72*time.Hour {
+		t.Errorf("available in %s, want ~72h", d)
+	}
+}
+
+func TestQuarantineRedisErrorUnavailable(t *testing.T) {
+	requireStack(t)
+	ctx := context.Background()
+	r := quarantineRemote(t, "q-redis-down", true)
+	r.StaleOnError = true
+	down := must(cache.NewRedis(testRedis))
+	_ = down.Close()
+	eng := NewEngine(testDB, down, testEngine.store)
+	_, err := eng.Fetch(ctx, r, "undated.tgz", prov(t, models.PackageNPM))
+	var pe *ProxyError
+	if !errors.As(err, &pe) || pe.Status != http.StatusServiceUnavailable || pe.RetryAfter == 0 || Quarantined(err) {
+		t.Fatalf("want 503 with Retry-After, got %#v", err)
+	}
+	if a, _ := testDB.GetArtifact(ctx, r.Name, "undated.tgz"); a != nil {
+		t.Error("artifact cached while quarantine check unavailable")
+	}
+}
+
+func TestQuarantineStaleOnErrorServesNothingUncached(t *testing.T) {
+	requireStack(t)
+	ctx := context.Background()
+	r := quarantineRemote(t, "q-stale", true)
+	r.StaleOnError = true
+	p := prov(t, models.PackageNPM)
+	_, err := testEngine.Fetch(ctx, r, "fresh.tgz", p)
+	_ = wantQuarantined(t, err)
+	r.BaseURL = "http://127.0.0.1:1"
+	if _, err := testEngine.Fetch(ctx, r, "fresh.tgz", p); err == nil {
+		t.Error("stale_on_error served an uncached quarantined path")
+	}
+	if a, _ := testDB.GetArtifact(ctx, r.Name, "fresh.tgz"); a != nil {
+		t.Error("quarantined artifact was cached")
+	}
+}

internal/server/oidc_test.go

@@ -1,6 +1,7 @@
 package server
 
 import (
+	"encoding/base64"
 	"io"
 	"net/http"
 	"net/http/httptest"
@@ -78,3 +79,98 @@ func TestServerOIDCWriteAuth(t *testing.T) {
 	expect(204, "DELETE", "/api/v2/virtuals/srv-oidc-v", "", good)
 	expect(204, "DELETE", "/api/v2/remotes/srv-oidc", "", good)
 }
+
+func TestServerAuthUploads(t *testing.T) {
+	requireStack(t)
+	iss := testsupport.NewOIDCIssuer()
+	defer iss.Close()
+
+	cfg := *testSrv.cfg
+	cfg.OIDCIssuer = iss.URL
+	cfg.OIDCAudiences = []string{"artifactapi"}
+	cfg.AuthUploads = true
+	s := *testSrv
+	s.cfg = &cfg
+	ts := httptest.NewServer(s.routes())
+	defer ts.Close()
+
+	good := iss.Token(nil, "artifactapi", "ci", time.Hour)
+	do := func(method, path, body, authz string) int {
+		t.Helper()
+		rq, _ := http.NewRequest(method, ts.URL+path, strings.NewReader(body))
+		rq.Header.Set("Content-Type", "application/json")
+		if authz != "" {
+			rq.Header.Set("Authorization", authz)
+		}
+		resp, err := http.DefaultClient.Do(rq)
+		if err != nil {
+			t.Fatalf("%s %s: %v", method, path, err)
+		}
+		_, _ = io.Copy(io.Discard, resp.Body)
+		_ = resp.Body.Close()
+		return resp.StatusCode
+	}
+	bearer := "Bearer " + good
+	for _, typ := range []string{"generic", "docker", "npm", "gems"} {
+		body := `{"name":"srv-up-` + typ + `","package_type":"` + typ + `","repo_type":"local"}`
+		if got := do("POST", "/api/v2/remotes", body, bearer); got != 201 {
+			t.Fatalf("create %s: status %d", typ, got)
+		}
+	}
+
+	writes := []struct{ method, path, body string }{
+		{"PUT", "/api/v2/remotes/srv-up-generic/files/a.bin", "payload"},
+		{"DELETE", "/api/v2/remotes/srv-up-generic/files/a.bin", ""},
+		{"DELETE", "/api/v2/remotes/srv-up-generic/objects/a.bin", ""},
+		{"DELETE", "/api/v2/locals/srv-up-generic/objects/a.bin", ""},
+		{"PUT", "/api/v1/local/srv-up-npm/pkg", "{}"},
+		{"POST", "/api/v1/local/srv-up-gems/api/v1/gems", "x"},
+		{"DELETE", "/api/v1/local/srv-up-gems/api/v1/gems/yank", ""},
+		{"POST", "/v2/srv-up-docker/img/blobs/uploads/", ""},
+		{"PATCH", "/v2/srv-up-docker/img/blobs/uploads/x", ""},
+		{"PUT", "/v2/srv-up-docker/img/manifests/t", "{}"},
+		{"DELETE", "/v2/srv-up-docker/img/manifests/t", ""},
+	}
+	for _, w := range writes {
+		if got := do(w.method, w.path, w.body, ""); got != 401 {
+			t.Errorf("anonymous %s %s: status %d, want 401", w.method, w.path, got)
+		}
+	}
+
+	if got := do("PUT", "/api/v2/remotes/srv-up-generic/files/a.bin", "payload", bearer); got != 201 {
+		t.Errorf("bearer upload: status %d", got)
+	}
+	if got := do("PUT", "/api/v2/remotes/srv-up-generic/files/b.bin", "payload", basic(good)); got != 201 {
+		t.Errorf("basic upload: status %d", got)
+	}
+	if got := do("POST", "/v2/srv-up-docker/img/blobs/uploads/", "", basic(good)); got != 202 {
+		t.Errorf("basic docker upload: status %d", got)
+	}
+	expectOpen := func(method, path string) {
+		t.Helper()
+		if got := do(method, path, "", ""); got >= 400 {
+			t.Errorf("anonymous %s %s: status %d", method, path, got)
+		}
+	}
+	expectOpen("GET", "/api/v2/remotes/srv-up-generic/files/a.bin")
+	expectOpen("GET", "/api/v2/remotes/srv-up-generic/objects")
+	expectOpen("GET", "/api/v1/local/srv-up-generic/a.bin")
+	expectOpen("GET", "/v2/token")
+	if got := do("GET", "/v2/", "", ""); got != 401 {
+		t.Errorf("anonymous ping: status %d, want 401 challenge", got)
+	}
+	if got := do("POST", "/api/v2/probe", `{}`, ""); got == 401 {
+		t.Error("probe gated")
+	}
+
+	if got := do("DELETE", "/api/v2/remotes/srv-up-generic/files/a.bin", "", bearer); got >= 400 {
+		t.Errorf("bearer delete: status %d", got)
+	}
+	for _, typ := range []string{"generic", "docker", "npm", "gems"} {
+		do("DELETE", "/api/v2/remotes/srv-up-"+typ, "", bearer)
+	}
+}
+
+func basic(tok string) string {
+	return "Basic " + base64.StdEncoding.EncodeToString([]byte("ci:"+tok))
+}

internal/server/server.go

@@ -169,8 +169,17 @@ func (s *Server) routes() chi.Router {
 	s.virtEngine.NPMLocal = proxyHandler.LocalNPMPackument
 	s.virtEngine.GemsLocal = proxyHandler.LocalGemsIndex
 	s.virtEngine.AnsibleLocal = proxyHandler.LocalAnsible
-	r.Mount("/api/v1", proxyHandler.Routes())
+	writeAuth := auth.NewBearerOIDC(s.cfg.OIDCIssuer, s.cfg.OIDCAudiences)
-	r.Mount("/v2", proxyHandler.DockerV2Routes())
+	uploadAuth := passthrough
+	dockerV2 := http.Handler(proxyHandler.DockerV2Routes())
+	if s.cfg.AuthUploads {
+		uploadAuth = writeAuth.RequireWrites
+		dockerV2 = writeAuth.RequireRegistryWrites(dockerV2, func(r *http.Request) string {
+			return v1.Scheme(r) + "://" + r.Host + "/v2/token"
+		})
+	}
+	r.Mount("/api/v1", uploadAuth(proxyHandler.Routes()))
+	r.Mount("/v2", dockerV2)
 
 	remotesHandler := v2.NewRemotesHandler(s.db, s.cache, map[models.PackageType]v2.Primer{
 		models.PackageGitHubRPM:    s.syncer,
@@ -182,7 +191,6 @@ func (s *Server) routes() chi.Router {
 	statsHandler := v2.NewStatsHandler(s.db)
 	eventsHandler := v2.NewEventsHandler()
 	probeHandler := v2.NewProbeHandler(s.engine, s.db)
-	writeAuth := auth.NewBearerOIDC(s.cfg.OIDCIssuer, s.cfg.OIDCAudiences)
 
 	r.Route("/api/v2", func(r chi.Router) {
 		r.Mount("/remotes", writeAuth.RequireWrites(remotesHandler.Routes()))
@@ -192,19 +200,19 @@ func (s *Server) routes() chi.Router {
 		r.Mount("/events", eventsHandler.Routes())
 		r.Mount("/probe", probeHandler.Routes())
 
-		r.Route("/remotes/{name}/objects", func(r chi.Router) {
+		r.With(uploadAuth).Route("/remotes/{name}/objects", func(r chi.Router) {
 			objHandler := v2.NewObjectsHandler(s.db)
 			r.Get("/", objHandler.Routes(s.engine).ServeHTTP)
 			r.Delete("/*", objHandler.Routes(s.engine).ServeHTTP)
 		})
 
-		r.Route("/locals/{name}/objects", func(r chi.Router) {
+		r.With(uploadAuth).Route("/locals/{name}/objects", func(r chi.Router) {
 			objHandler := v2.NewObjectsHandler(s.db)
 			r.Get("/", objHandler.LocalRoutes().ServeHTTP)
 			r.Delete("/*", objHandler.LocalRoutes().ServeHTTP)
 		})
 
-		r.Route("/remotes/{name}/files", func(r chi.Router) {
+		r.With(uploadAuth).Route("/remotes/{name}/files", func(r chi.Router) {
 			r.Put("/*", s.localHandler.Routes().ServeHTTP)
 			r.Get("/*", s.localHandler.Routes().ServeHTTP)
 			r.Delete("/*", s.localHandler.Routes().ServeHTTP)
@@ -214,6 +222,8 @@ func (s *Server) routes() chi.Router {
 	return r
 }
 
+func passthrough(h http.Handler) http.Handler { return h }
+
 func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "application/json")
 	w.WriteHeader(http.StatusOK)

internal/server/server_test.go

@@ -416,6 +416,25 @@ func TestServerRemoteUpdateAndVirtualCRUD(t *testing.T) {
 	}
 }
 
+func TestServerRemoteServeStaleRoundTrip(t *testing.T) {
+	requireStack(t)
+	if resp, b := req(t, "POST", "/api/v2/remotes", `{"name":"srv-ss","package_type":"rpm","repo_type":"remote","base_url":"https://a.example.com","serve_stale":true}`); resp.StatusCode != 201 {
+		t.Fatalf("create remote: %d %s", resp.StatusCode, b)
+	}
+	defer req(t, "DELETE", "/api/v2/remotes/srv-ss", "")
+	get := func(want string) {
+		t.Helper()
+		if _, b := req(t, "GET", "/api/v2/remotes/srv-ss", ""); !strings.Contains(string(b), want) {
+			t.Errorf("get remote: want %s in %s", want, b)
+		}
+	}
+	get(`"serve_stale":true`)
+	if resp, b := req(t, "PUT", "/api/v2/remotes/srv-ss", `{"package_type":"rpm","base_url":"https://a.example.com"}`); resp.StatusCode != 200 {
+		t.Fatalf("update remote: %d %s", resp.StatusCode, b)
+	}
+	get(`"serve_stale":false`)
+}
+
 func TestServerLocalRemoveAndMissing(t *testing.T) {
 	requireStack(t)
 	req(t, "POST", "/api/v2/remotes", `{"name":"srv-rm","package_type":"generic","repo_type":"local"}`)

internal/tui/render.go

@@ -58,6 +58,8 @@ func (m *model) viewRemoteDetail() string {
 	sb.WriteString(fmt.Sprintf("  Mutable TTL:    %ds\n", r.MutableTTL))
 	sb.WriteString(fmt.Sprintf("  Revalidation:   %v\n", r.CheckMutable))
 	sb.WriteString(fmt.Sprintf("  Stale on Error: %v\n", r.StaleOnError))
+	fmt.Fprintf(&sb, "  Serve Stale:    %v\n", r.ServeStale)
+	fmt.Fprintf(&sb, "  Cache Repodata: %v\n", r.CacheRepodata)
 
 	if len(r.Patterns) > 0 {
 		sb.WriteString(fmt.Sprintf("  Patterns:       %s\n", strings.Join(r.Patterns, ", ")))
@@ -107,8 +109,12 @@ func (m *model) viewVirtualsList() string {
 	}
 
 	for i, v := range m.virtuals {
-		line := fmt.Sprintf("  %-25s %-12s %d members  %s",
+		parallel := "-"
-			v.Name, v.PackageType, len(v.Members), v.Description)
+		if v.ParallelFetch {
+			parallel = "parallel"
+		}
+		line := fmt.Sprintf("  %-25s %-12s %3d members  %-8s  %s",
+			v.Name, v.PackageType, len(v.Members), parallel, v.Description)
 		if i == m.cursor {
 			sb.WriteString(selStyle.Render(line))
 		} else {

internal/virtual/alpine_merger.go

@@ -0,0 +1,90 @@
+package virtual
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"errors"
+	"fmt"
+	"io"
+	"strings"
+	"time"
+)
+
+// AlpineMember is one member's decompressed APKINDEX for a directory; Index is
+// nil when the member does not serve that directory.
+type AlpineMember struct {
+	RemoteName string
+	Index      []byte
+}
+
+// AlpineRepo is a merged directory: its APKINDEX.tar.gz plus, per
+// "<name>-<version>.apk" apk will request, the member that owns it.
+type AlpineRepo struct {
+	Index  []byte
+	Owners map[string][]byte
+}
+
+func (r *AlpineRepo) index() []byte            { return r.Index }
+func (r *AlpineRepo) files() map[string][]byte { return r.Owners }
+
+// MergeAlpine merges members' APKINDEX records. Members are in priority order:
+// the first to carry a (P, V, A) wins. The index is unsigned, like the local
+// and github_alpine indexes it merges.
+func MergeAlpine(members []AlpineMember) *AlpineRepo {
+	repo := &AlpineRepo{Owners: map[string][]byte{}}
+	seen := map[string]bool{}
+	var b bytes.Buffer
+	for _, m := range members {
+		for _, rec := range splitStanzas(m.Index) {
+			p, v, a := stanzaField(rec, "P"), stanzaField(rec, "V"), stanzaField(rec, "A")
+			key := p + "\x00" + v + "\x00" + a
+			if p == "" || v == "" || seen[key] {
+				continue
+			}
+			seen[key] = true
+			if _, ok := repo.Owners[p+"-"+v+".apk"]; !ok {
+				repo.Owners[p+"-"+v+".apk"] = []byte(m.RemoteName)
+			}
+			b.WriteString(rec)
+			b.WriteString("\n\n")
+		}
+	}
+	var t bytes.Buffer
+	tw := tar.NewWriter(&t)
+	_ = tw.WriteHeader(&tar.Header{Name: "APKINDEX", Mode: 0o644, Size: int64(b.Len()), Typeflag: tar.TypeReg, ModTime: time.Unix(0, 0)})
+	_, _ = tw.Write(b.Bytes())
+	_ = tw.Close()
+	repo.Index = gzipDeterministic(t.Bytes())
+	return repo
+}
+
+// apkIndexText extracts the APKINDEX entry from an APKINDEX.tar.gz, skipping a
+// leading signature stream if present.
+func apkIndexText(body []byte) ([]byte, error) {
+	zr, err := gzip.NewReader(bytes.NewReader(body))
+	if err != nil {
+		return nil, err
+	}
+	tr := tar.NewReader(zr)
+	for {
+		hdr, err := tr.Next()
+		if errors.Is(err, io.EOF) {
+			return nil, fmt.Errorf("no APKINDEX entry")
+		}
+		if err != nil {
+			return nil, err
+		}
+		if hdr.Name == "APKINDEX" {
+			return io.ReadAll(tr)
+		}
+	}
+}
+
+// alpinePath splits a request into its directory and file, or ok is false for
+// anything but an APKINDEX.tar.gz or .apk.
+func alpinePath(p string) (dir, file string, ok bool) {
+	i := strings.LastIndex(p, "/")
+	dir, file = p[:i+1], p[i+1:]
+	return dir, file, dir != "" && (file == "APKINDEX.tar.gz" || strings.HasSuffix(file, ".apk"))
+}

internal/virtual/alpine_merger_test.go

@@ -0,0 +1,140 @@
+package virtual
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"context"
+	"errors"
+	"strings"
+	"testing"
+
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+func apkRec(p, v, a string) string {
+	return "C:Q1" + p + v + "\nP:" + p + "\nV:" + v + "\nA:" + a + "\nS:10\n"
+}
+
+func apkTarGz(t *testing.T, name string, body []byte, closeTar bool) []byte {
+	t.Helper()
+	var tb bytes.Buffer
+	tw := tar.NewWriter(&tb)
+	if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o644, Size: int64(len(body)), Typeflag: tar.TypeReg}); err != nil {
+		t.Fatal(err)
+	}
+	tw.Write(body)
+	if closeTar {
+		tw.Close()
+	} else {
+		tw.Flush()
+	}
+	var gb bytes.Buffer
+	gz := gzip.NewWriter(&gb)
+	gz.Write(tb.Bytes())
+	gz.Close()
+	return gb.Bytes()
+}
+
+func TestMergeAlpine(t *testing.T) {
+	repo := MergeAlpine([]AlpineMember{
+		{RemoteName: "a", Index: []byte(apkRec("foo", "1.0-r0", "x86_64") + "\n" + apkRec("bar", "1-r0", "noarch") + "\nC:Q1x\nV:9\n")},
+		{RemoteName: "b"},
+		{RemoteName: "c", Index: []byte(apkRec("foo", "1.0-r0", "x86_64") + "\n" + apkRec("foo", "2.0-r0", "x86_64") + "\n" + apkRec("bar", "1-r0", "x86_64"))},
+	})
+	idx, err := apkIndexText(repo.Index)
+	if err != nil {
+		t.Fatal(err)
+	}
+	recs := splitStanzas(idx)
+	if len(recs) != 4 || strings.Count(string(idx), "P:foo\nV:1.0-r0\n") != 1 {
+		t.Fatalf("want foo 1.0 once, foo 2.0 and bar for both arches, record without P dropped:\n%s", idx)
+	}
+	for file, want := range map[string]string{"foo-1.0-r0.apk": "a", "foo-2.0-r0.apk": "c", "bar-1-r0.apk": "a"} {
+		if got := string(repo.Owners[file]); got != want {
+			t.Errorf("%s owned by %q, want %q", file, got, want)
+		}
+	}
+	if again := MergeAlpine([]AlpineMember{{RemoteName: "a", Index: idx}}); !bytes.Equal(again.Index, MergeAlpine([]AlpineMember{{RemoteName: "a", Index: idx}}).Index) {
+		t.Fatal("merged index must be byte-stable across merges")
+	}
+}
+
+func TestApkIndexTextSkipsSignature(t *testing.T) {
+	signed := append(apkTarGz(t, ".SIGN.RSA.alpine.rsa.pub", []byte("sig"), false), apkTarGz(t, "APKINDEX", []byte(apkRec("foo", "1", "x86_64")), true)...)
+	idx, err := apkIndexText(signed)
+	if err != nil || !strings.Contains(string(idx), "P:foo\n") {
+		t.Fatalf("signed index: %v\n%s", err, idx)
+	}
+	if _, err := apkIndexText(apkTarGz(t, "DESCRIPTION", []byte("x"), true)); err == nil {
+		t.Fatal("index without APKINDEX entry should fail")
+	}
+}
+
+func alpineEngine(members map[string]string) *Engine {
+	e := fakeEngine(nil)
+	e.apkMember = func(_ context.Context, name, dir string) (*AlpineMember, error) {
+		idx, ok := members[name]
+		if !ok {
+			return nil, errors.New("upstream down")
+		}
+		if dir != "x86_64/" {
+			return &AlpineMember{RemoteName: name}, nil
+		}
+		return &AlpineMember{RemoteName: name, Index: []byte(idx)}, nil
+	}
+	return e
+}
+
+func alpineVirt(members ...string) models.Virtual {
+	return models.Virtual{Name: "v", PackageType: models.PackageAlpine, Members: members}
+}
+
+func TestFetchAlpineIndexAndRedirect(t *testing.T) {
+	e := alpineEngine(map[string]string{"loc": apkRec("foo", "1-r0", "x86_64"), "gh": apkRec("foo", "1-r0", "x86_64") + "\n" + apkRec("tool", "2-r0", "x86_64")})
+	virt := alpineVirt("loc", "gh")
+
+	body, ct, err := e.Fetch(context.Background(), virt, "./x86_64/APKINDEX.tar.gz", "")
+	if err != nil || ct != "application/gzip" {
+		t.Fatalf("fetch index: %v %q", err, ct)
+	}
+	if idx, _ := apkIndexText(body); strings.Count(string(idx), "P:") != 2 {
+		t.Fatalf("merged index:\n%s", idx)
+	}
+	for _, tc := range []struct {
+		path, want string
+		ok         bool
+	}{
+		{path: "x86_64/foo-1-r0.apk", ok: true, want: "http://h/api/v1/remote/loc/x86_64/foo-1-r0.apk"},
+		{path: "./x86_64/tool-2-r0.apk", ok: true, want: "http://h/api/v1/remote/gh/x86_64/tool-2-r0.apk"},
+		{path: "x86_64/../x86_64/tool-2-r0.apk", ok: true, want: "http://h/api/v1/remote/gh/x86_64/tool-2-r0.apk"},
+		{path: "x86_64/missing-1-r0.apk"},
+		{path: "aarch64/foo-1-r0.apk"},
+		{path: "x86_64/APKINDEX.tar.gz"},
+		{path: "foo-1-r0.apk"},
+	} {
+		got, ok, err := e.MemberRedirect(context.Background(), virt, tc.path, "", "http://h/")
+		if got != tc.want || ok != tc.ok || err != nil {
+			t.Errorf("%s: got (%q, %v, %v), want (%q, %v)", tc.path, got, ok, err, tc.want, tc.ok)
+		}
+	}
+	for _, p := range []string{"x86_64/missing-1-r0.apk", "x86_64/DESCRIPTION", "APKINDEX.tar.gz"} {
+		if _, _, err := e.Fetch(context.Background(), virt, p, ""); !errors.Is(err, ErrNotFound) {
+			t.Errorf("%s: want not found, got %v", p, err)
+		}
+	}
+}
+
+func TestFetchAlpineFailsClosed(t *testing.T) {
+	e := alpineEngine(map[string]string{"a": apkRec("foo", "1-r0", "x86_64")})
+	virt := alpineVirt("a", "down")
+	if _, _, err := e.Fetch(context.Background(), virt, "x86_64/APKINDEX.tar.gz", ""); err == nil || errors.Is(err, ErrNotFound) {
+		t.Fatalf("index with a member down: want upstream error, got %v", err)
+	}
+	if _, ok, err := e.MemberRedirect(context.Background(), virt, "x86_64/foo-1-r0.apk", "", "http://h/"); ok || err != nil {
+		t.Fatalf("redirect with a member down must defer to Fetch: ok %v err %v", ok, err)
+	}
+	if _, _, err := e.Fetch(context.Background(), virt, "x86_64/foo-1-r0.apk", ""); err == nil || errors.Is(err, ErrNotFound) {
+		t.Fatalf("download with a member down: want upstream error, got %v", err)
+	}
+}

internal/virtual/engine.go

@@ -19,6 +19,7 @@ import (
 	"git.unkin.net/unkin/artifactapi/internal/provider"
 	"git.unkin.net/unkin/artifactapi/internal/proxy"
 	"git.unkin.net/unkin/artifactapi/pkg/models"
+	"golang.org/x/sync/errgroup"
 	"golang.org/x/sync/singleflight"
 )
 
@@ -26,9 +27,11 @@ type Engine struct {
 	db          *database.DB
 	proxyEngine *proxy.Engine
 	getRemote   func(context.Context, string) (*models.Remote, error)
-	rpmMember   func(context.Context, string) (*RPMMember, error)
+	rpmMember   func(ctx context.Context, name string, parallel bool) (*RPMMember, error)
-	debMember   func(ctx context.Context, name, base string) (*DebMember, error)
+	debMember   func(ctx context.Context, name, base string, parallel bool) (*DebMember, error)
+	apkMember   func(ctx context.Context, name, dir string) (*AlpineMember, error)
 	npmMember   func(ctx context.Context, name, pkg string) (*NPMMember, error)
+	memberPath  func(ctx context.Context, remote models.Remote, path string) ([]byte, error)
 
 	// NPMLocal returns a local npm repo's packument with tarball paths relative
 	// to the repo, or ErrNotFound.
@@ -54,8 +57,20 @@ type Engine struct {
 	sf   singleflight.Group
 	mu   sync.Mutex
 	gens map[string]*repoGen
+	// stale holds the last good fetch of each serve_stale member, keyed by
+	// merge key and member name.
+	stale map[string]*staleEntry
 }
 
+type staleEntry struct {
+	m  any
+	at time.Time
+}
+
+// maxStaleAge bounds how old a serve_stale member copy may be before merges
+// block on the member again and surface its error.
+const maxStaleAge = time.Hour
+
 // mergedRepo is a merged index document plus the files it references.
 type mergedRepo interface {
 	index() []byte
@@ -76,18 +91,21 @@ type repoGen struct {
 
 const rpmMergeTTL = 60 * time.Second
 
-// maxDebSuites bounds the client-chosen suites cached per deb virtual.
+// maxDebSuites bounds the client-chosen deb suites or alpine directories
+// cached per virtual.
 const maxDebSuites = 32
 
 func NewEngine(db *database.DB, proxyEngine *proxy.Engine) *Engine {
 	e := &Engine{db: db, proxyEngine: proxyEngine, getRemote: db.GetRemote, mergeTTL: rpmMergeTTL}
 	e.rpmMember = e.fetchRPMMember
 	e.debMember = e.fetchDebMember
+	e.apkMember = e.fetchAlpineMember
 	e.npmMember = e.fetchNPMMember
 	e.gemsMember = e.fetchGemsMember
 	e.ansibleDoc = e.ansibleMemberDoc
 	e.goMember = e.fetchGoMember
 	e.puppetDoc = e.fetchMemberPath
+	e.memberPath = e.fetchMemberPath
 	return e
 }
 
@@ -97,6 +115,8 @@ func (e *Engine) Fetch(ctx context.Context, virt models.Virtual, path string, pr
 		return e.fetchRPM(ctx, virt, path)
 	case models.PackageDeb:
 		return e.fetchDeb(ctx, virt, path)
+	case models.PackageAlpine:
+		return e.fetchAlpine(ctx, virt, path)
 	}
 
 	merger, err := GetMerger(virt.PackageType)
@@ -216,8 +236,8 @@ var (
 )
 
 // MemberRedirect maps an rpm, deb, npm or ansible virtual package path
-// (<member>/<href>, as written by MergeRPM/MergeDeb/MergeNPM/AnsibleVersion)
+// (<member>/<href>, as written by MergeRPM/MergeDeb/MergeNPM/AnsibleVersion),
-// to the owning member's route. ok is false for any other path; ErrBadPath
+// or an alpine <dir>/<name>-<version>.apk, to the owning member's route. ok is false for any other path; ErrBadPath
 // means the path is absolute or escapes the member.
 func (e *Engine) MemberRedirect(ctx context.Context, virt models.Virtual, path, rawQuery, proxyBaseURL string) (string, bool, error) {
 	switch virt.PackageType {
@@ -231,6 +251,8 @@ func (e *Engine) MemberRedirect(ctx context.Context, virt models.Virtual, path,
 		if strings.HasPrefix(path, "dists/") {
 			return "", false, nil
 		}
+	case models.PackageAlpine:
+		return e.alpineRedirect(ctx, virt, path, rawQuery, proxyBaseURL)
 	case models.PackageNPM, models.PackageAnsible:
 	default:
 		return "", false, nil
@@ -318,6 +340,7 @@ func (e *Engine) merged(ctx context.Context, key string, merge func(context.Cont
 	e.mu.Unlock()
 
 	v, err, _ := e.sf.Do(key, func() (any, error) {
+		e.pruneStale()
 		repo, err := merge(context.WithoutCancel(ctx))
 		if err != nil {
 			return nil, err
@@ -346,7 +369,9 @@ func (e *Engine) merged(ctx context.Context, key string, merge func(context.Cont
 }
 
 func (e *Engine) mergeRPM(ctx context.Context, virt models.Virtual) (*RPMRepo, error) {
-	members, err := fetchMembers(virt, func(name string) (*RPMMember, error) { return e.rpmMember(ctx, name) })
+	members, err := fetchMembers(virt, func(name string) (*RPMMember, error) {
+		return staleMember(ctx, e, "rpm\x00"+virt.Name, name, func(ctx context.Context) (*RPMMember, error) { return e.rpmMember(ctx, name, virt.ParallelFetch) })
+	})
 	if err != nil {
 		return nil, err
 	}
@@ -395,7 +420,9 @@ func (e *Engine) fetchDeb(ctx context.Context, virt models.Virtual, reqPath stri
 	}
 
 	repo, err := e.merged(ctx, key, func(ctx context.Context) (mergedRepo, error) {
-		members, err := fetchMembers(virt, func(name string) (*DebMember, error) { return e.debMember(ctx, name, base) })
+		members, err := fetchMembers(virt, func(name string) (*DebMember, error) {
+			return staleMember(ctx, e, key, name, func(ctx context.Context) (*DebMember, error) { return e.debMember(ctx, name, base, virt.ParallelFetch) })
+		})
 		if err != nil {
 			return nil, err
 		}
@@ -423,26 +450,165 @@ func debContentType(name string) string {
 
 // fetchDebMember reads a member's Release under base and each binary Packages
 // index it lists.
-func (e *Engine) fetchDebMember(ctx context.Context, name, base string) (*DebMember, error) {
+func (e *Engine) fetchDebMember(ctx context.Context, name, base string, parallel bool) (*DebMember, error) {
 	remote, err := e.getRemote(ctx, name)
 	if err != nil {
 		return nil, fmt.Errorf("remote %q: %w", name, err)
 	}
-	rel, err := e.fetchMemberPath(ctx, *remote, base+"Release")
+	rel, err := e.memberPath(ctx, *remote, base+"Release")
 	if err != nil {
 		return nil, err
 	}
-	m := &DebMember{RemoteName: name, Release: rel, Packages: map[string][]byte{}}
+	pkgs, err := fetchFiles(ctx, parallel, debIndexFiles(parseDebRelease(rel)), func(ctx context.Context, file string) ([]byte, error) {
-	for dir, file := range debIndexFiles(parseDebRelease(rel)) {
+		return e.fetchDecompressed(ctx, *remote, base, file)
-		raw, err := e.fetchMemberPath(ctx, *remote, base+file)
+	})
+	if err != nil {
+		return nil, err
+	}
+	return &DebMember{RemoteName: name, Release: rel, Packages: pkgs}, nil
+}
+
+func (e *Engine) mergedAlpine(ctx context.Context, virt models.Virtual, dir string) (mergedRepo, error) {
+	key := "alpine\x00" + virt.Name + "\x00" + dir
+	repo, err := e.merged(ctx, key, func(ctx context.Context) (mergedRepo, error) {
+		members, err := fetchMembers(virt, func(name string) (*AlpineMember, error) {
+			return staleMember(ctx, e, key, name, func(ctx context.Context) (*AlpineMember, error) { return e.apkMember(ctx, name, dir) })
+		})
 		if err != nil {
 			return nil, err
 		}
-		if m.Packages[dir], err = decompress(file, raw); err != nil {
+		return MergeAlpine(members), nil
-			return nil, fmt.Errorf("decompress %s: %w", file, err)
+	})
+	if err == nil {
+		e.evictLRU("alpine\x00"+virt.Name+"\x00", maxDebSuites)
+	}
+	return repo, err
+}
+
+func (e *Engine) fetchAlpine(ctx context.Context, virt models.Virtual, reqPath string) ([]byte, string, error) {
+	dir, file, ok := alpinePath(strings.TrimPrefix(path.Clean("/"+reqPath), "/"))
+	if !ok {
+		return nil, "", ErrNotFound
+	}
+	repo, err := e.mergedAlpine(ctx, virt, dir)
+	if err != nil {
+		return nil, "", err
+	}
+	if file != "APKINDEX.tar.gz" {
+		return nil, "", ErrNotFound
+	}
+	return repo.index(), "application/gzip", nil
+}
+
+// alpineRedirect finds the member whose index lists the requested .apk; apk
+// builds the download path itself, so the index cannot carry a member prefix.
+// A failed merge returns ok=false so Fetch re-merges and fails with 502.
+func (e *Engine) alpineRedirect(ctx context.Context, virt models.Virtual, reqPath, rawQuery, proxyBaseURL string) (string, bool, error) {
+	dir, file, ok := alpinePath(strings.TrimPrefix(path.Clean("/"+reqPath), "/"))
+	if !ok || file == "APKINDEX.tar.gz" {
+		return "", false, nil
+	}
+	owner, ok := e.cachedFile("alpine\x00"+virt.Name+"\x00"+dir, file)
+	if !ok {
+		repo, err := e.mergedAlpine(ctx, virt, dir)
+		if err != nil {
+			return "", false, nil
+		}
+		if owner, ok = repo.files()[file]; !ok {
+			return "", false, nil
+		}
+	}
+	return e.memberLocation(ctx, string(owner), dir+file, rawQuery, proxyBaseURL)
+}
+
+// fetchAlpineMember reads a member's APKINDEX under dir; a member without that
+// directory contributes nothing.
+func (e *Engine) fetchAlpineMember(ctx context.Context, name, dir string) (*AlpineMember, error) {
+	remote, err := e.getRemote(ctx, name)
+	if err != nil {
+		return nil, fmt.Errorf("remote %q: %w", name, err)
+	}
+	raw, err := e.fetchMemberPath(ctx, *remote, dir+"APKINDEX.tar.gz")
+	var pe *proxy.ProxyError
+	if errors.Is(err, ErrNotFound) || errors.As(err, &pe) && pe.Status == http.StatusNotFound {
+		return &AlpineMember{RemoteName: name}, nil
+	}
+	if err != nil {
+		return nil, err
+	}
+	idx, err := apkIndexText(raw)
+	if err != nil {
+		return nil, fmt.Errorf("%s/%sAPKINDEX.tar.gz: %w", name, dir, err)
+	}
+	return &AlpineMember{RemoteName: name, Index: idx}, nil
+}
+
+// staleMember fetches member name for the merge under key. A serve_stale member
+// with a good fetch younger than maxStaleAge returns it at once and refreshes in
+// the background, one refresh per member at a time; the next merge picks it up.
+func staleMember[T any](ctx context.Context, e *Engine, key, name string, fetch func(context.Context) (*T, error)) (*T, error) {
+	sk := key + "\x00" + name
+	remote, lookupErr := e.getRemote(ctx, name)
+	if lookupErr == nil && !remote.ServeStale {
+		e.mu.Lock()
+		delete(e.stale, sk)
+		e.mu.Unlock()
+		return fetch(ctx)
+	}
+	e.mu.Lock()
+	if e.stale == nil {
+		e.stale = map[string]*staleEntry{}
+	}
+	prev := e.stale[sk]
+	// A failed lookup serves an existing copy but never creates one.
+	if lookupErr != nil && (prev == nil || prev.m == nil) {
+		e.mu.Unlock()
+		return fetch(ctx)
+	}
+	// A cold fetch holds a placeholder so eviction or toggle-off during it is seen.
+	token := prev
+	if prev == nil || prev.m == nil {
+		prev, token = nil, &staleEntry{at: time.Now()}
+		e.stale[sk] = token
+	}
+	e.mu.Unlock()
+	refresh := func() (any, error) {
+		m, err := fetch(ctx)
+		e.mu.Lock()
+		// An evicted or replaced entry means this refresh is no longer wanted.
+		if e.stale[sk] == token {
+			if err == nil {
+				e.stale[sk] = &staleEntry{m: m, at: time.Now()}
+			} else if prev == nil {
+				delete(e.stale, sk)
+			}
+		}
+		e.mu.Unlock()
+		return m, err
+	}
+	if prev == nil {
+		m, err := refresh()
+		return m.(*T), err
+	}
+	ch := e.sf.DoChan("stale\x00"+sk, refresh)
+	go func() {
+		if r := <-ch; r.Err != nil {
+			slog.Warn("virtual member background refresh failed", "member", name, "error", r.Err)
+		}
+	}()
+	return prev.m.(*T), nil
+}
+
+// pruneStale drops serve_stale copies past maxStaleAge, including those of
+// removed members and virtuals.
+func (e *Engine) pruneStale() {
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	for k, se := range e.stale {
+		if time.Since(se.at) >= maxStaleAge {
+			delete(e.stale, k)
 		}
 	}
-	return m, nil
 }
 
 // evictLRU drops the least recently used merges under prefix beyond keep.
@@ -458,6 +624,11 @@ func (e *Engine) evictLRU(prefix string, keep int) {
 	slices.SortFunc(keys, func(a, b string) int { return e.gens[b].used.Compare(e.gens[a].used) })
 	for _, k := range keys[min(len(keys), keep):] {
 		delete(e.gens, k)
+		for sk := range e.stale {
+			if strings.HasPrefix(sk, k+"\x00") {
+				delete(e.stale, sk)
+			}
+		}
 	}
 }
 
@@ -480,12 +651,12 @@ func (e *Engine) cachedFile(key, path string) ([]byte, bool) {
 	return nil, false
 }
 
-func (e *Engine) fetchRPMMember(ctx context.Context, name string) (*RPMMember, error) {
+func (e *Engine) fetchRPMMember(ctx context.Context, name string, parallel bool) (*RPMMember, error) {
 	remote, err := e.getRemote(ctx, name)
 	if err != nil {
 		return nil, fmt.Errorf("remote %q: %w", name, err)
 	}
-	repomd, err := e.fetchMemberPath(ctx, *remote, "repodata/repomd.xml")
+	repomd, err := e.memberPath(ctx, *remote, "repodata/repomd.xml")
 	if err != nil {
 		return nil, err
 	}
@@ -493,17 +664,66 @@ func (e *Engine) fetchRPMMember(ctx context.Context, name string) (*RPMMember, e
 	if err != nil {
 		return nil, err
 	}
-	m := &RPMMember{RemoteName: name, Bases: remote.UpstreamPool(), Timestamp: ts, Data: map[string][]byte{}}
+	data, err := fetchFiles(ctx, parallel, locs, func(ctx context.Context, href string) ([]byte, error) {
-	for t, href := range locs {
+		return e.fetchDecompressed(ctx, *remote, "", href)
-		raw, err := e.fetchMemberPath(ctx, *remote, href)
+	})
-		if err != nil {
+	if err != nil {
-			return nil, err
+		return nil, err
-		}
-		if m.Data[t], err = decompress(href, raw); err != nil {
-			return nil, fmt.Errorf("decompress %s: %w", href, err)
-		}
 	}
-	return m, nil
+	return &RPMMember{RemoteName: name, Bases: remote.UpstreamPool(), Timestamp: ts, Data: data}, nil
+}
+
+func (e *Engine) fetchDecompressed(ctx context.Context, remote models.Remote, base, file string) ([]byte, error) {
+	raw, err := e.memberPath(ctx, remote, base+file)
+	if err != nil {
+		return nil, err
+	}
+	body, err := decompress(file, raw)
+	if err != nil {
+		return nil, fmt.Errorf("decompress %s: %w", file, err)
+	}
+	return body, nil
+}
+
+// maxFileFetches bounds the concurrent index file fetches per member.
+const maxFileFetches = 4
+
+// fetchFiles fetches each key's file, one at a time or up to maxFileFetches at
+// once, failing on the first error and cancelling the rest.
+func fetchFiles(ctx context.Context, parallel bool, files map[string]string, fetch func(ctx context.Context, file string) ([]byte, error)) (map[string][]byte, error) {
+	out := make(map[string][]byte, len(files))
+	if !parallel {
+		for k, f := range files {
+			body, err := fetch(ctx, f)
+			if err != nil {
+				return nil, err
+			}
+			out[k] = body
+		}
+		return out, nil
+	}
+	var mu sync.Mutex
+	g, gctx := errgroup.WithContext(ctx)
+	g.SetLimit(maxFileFetches)
+	for k, f := range files {
+		g.Go(func() error {
+			if err := gctx.Err(); err != nil {
+				return err
+			}
+			body, err := fetch(gctx, f)
+			if err != nil {
+				return err
+			}
+			mu.Lock()
+			out[k] = body
+			mu.Unlock()
+			return nil
+		})
+	}
+	if err := g.Wait(); err != nil {
+		return nil, err
+	}
+	return out, nil
 }
 
 // fetchMemberPath reads one path from a member the same way its own route

internal/virtual/engine_stale_test.go

@@ -0,0 +1,465 @@
+package virtual
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+func rpmPkgMember(name, ver string) *RPMMember {
+	return &RPMMember{RemoteName: name, Data: map[string][]byte{"primary": primaryXML(primaryPkgXML("foo", ver, "h"+ver, "foo-"+ver+".rpm"))}}
+}
+
+// staleEngine re-merges on every request; member "a" is fetched through fetch
+// and has serve_stale set to stale.
+func staleEngine(stale bool, fetch func() *RPMMember) (*Engine, *atomic.Int32) {
+	e, calls, _ := staleEngineErr(stale, func() (*RPMMember, error) { return fetch(), nil })
+	return e, calls
+}
+
+func staleEngineErr(stale bool, fetch func() (*RPMMember, error)) (*Engine, *atomic.Int32, *atomic.Bool) {
+	var calls atomic.Int32
+	var flag atomic.Bool
+	flag.Store(stale)
+	e := fakeEngine(nil)
+	e.getRemote = func(_ context.Context, name string) (*models.Remote, error) {
+		return &models.Remote{Name: name, RepoType: models.RepoTypeRemote, ServeStale: flag.Load()}, nil
+	}
+	e.rpmMember = func(_ context.Context, name string, _ bool) (*RPMMember, error) {
+		calls.Add(1)
+		return fetch()
+	}
+	return e, &calls, &flag
+}
+
+// waitRefresh blocks until the background refresh of sk, if any, has finished.
+func waitRefresh(e *Engine, sk string) {
+	_, _, _ = e.sf.Do("stale\x00"+sk, func() (any, error) { return nil, nil })
+}
+
+const rpmStaleKey = "rpm\x00v\x00a"
+
+func fetchRepomd(t *testing.T, e *Engine) string {
+	t.Helper()
+	b, _, err := e.Fetch(context.Background(), rpmVirt("a"), "repodata/repomd.xml", "")
+	if err != nil {
+		t.Error(err)
+	}
+	return string(b)
+}
+
+func TestServeStaleMemberDoesNotBlockMerge(t *testing.T) {
+	release := make(chan struct{})
+	refreshing := make(chan struct{}, 1)
+	var slow atomic.Bool
+	e, calls := staleEngine(true, func() *RPMMember {
+		if slow.Load() {
+			select {
+			case refreshing <- struct{}{}:
+			default:
+			}
+			<-release
+			return rpmPkgMember("a", "2")
+		}
+		return rpmPkgMember("a", "1")
+	})
+	v1 := fetchRepomd(t, e)
+	slow.Store(true)
+
+	for range 3 {
+		done := make(chan string)
+		go func() { done <- fetchRepomd(t, e) }()
+		select {
+		case got := <-done:
+			if got != v1 {
+				t.Fatal("stale merge should serve the previous member copy")
+			}
+		case <-time.After(time.Second):
+			t.Fatal("merge blocked on a serve_stale member refresh")
+		}
+	}
+	select {
+	case <-refreshing:
+	case <-time.After(5 * time.Second):
+		t.Fatal("background refresh never started")
+	}
+	if n := calls.Load(); n != 2 {
+		t.Fatalf("member fetched %d times, want 2 (cold + one single-flighted refresh)", n)
+	}
+
+	close(release)
+	deadline := time.Now().Add(5 * time.Second)
+	for fetchRepomd(t, e) == v1 {
+		if time.Now().After(deadline) {
+			t.Fatal("background refresh never reached a merge")
+		}
+		time.Sleep(5 * time.Millisecond)
+	}
+}
+
+func assertMergeBlocks(t *testing.T, e *Engine, release chan struct{}) {
+	t.Helper()
+	done := make(chan struct{})
+	go func() { fetchRepomd(t, e); close(done) }()
+	select {
+	case <-done:
+		t.Fatal("merge returned before the member fetch finished")
+	case <-time.After(50 * time.Millisecond):
+	}
+	close(release)
+	<-done
+}
+
+func TestNonServeStaleMemberBlocksMerge(t *testing.T) {
+	release := make(chan struct{})
+	var slow atomic.Bool
+	e, _ := staleEngine(false, func() *RPMMember {
+		if slow.Load() {
+			<-release
+		}
+		return rpmPkgMember("a", "1")
+	})
+	fetchRepomd(t, e)
+	slow.Store(true)
+	assertMergeBlocks(t, e, release)
+}
+
+func TestServeStaleColdStartBlocksMerge(t *testing.T) {
+	release := make(chan struct{})
+	e, _ := staleEngine(true, func() *RPMMember {
+		<-release
+		return rpmPkgMember("a", "1")
+	})
+	assertMergeBlocks(t, e, release)
+}
+
+func TestServeStaleRefreshErrorKeepsCopy(t *testing.T) {
+	var down atomic.Bool
+	e, _, _ := staleEngineErr(true, func() (*RPMMember, error) {
+		if down.Load() {
+			return nil, errors.New("upstream down")
+		}
+		return rpmPkgMember("a", "1"), nil
+	})
+	v1 := fetchRepomd(t, e)
+	down.Store(true)
+	for range 3 {
+		if got := fetchRepomd(t, e); got != v1 {
+			t.Fatal("a failed refresh should keep serving the previous copy")
+		}
+		waitRefresh(e, rpmStaleKey)
+	}
+}
+
+func TestServeStaleMaxAgeFallsBackToFetch(t *testing.T) {
+	var down atomic.Bool
+	e, _, _ := staleEngineErr(true, func() (*RPMMember, error) {
+		if down.Load() {
+			return nil, errors.New("upstream down")
+		}
+		return rpmPkgMember("a", "1"), nil
+	})
+	fetchRepomd(t, e)
+	down.Store(true)
+	e.mu.Lock()
+	e.stale[rpmStaleKey].at = time.Now().Add(-maxStaleAge)
+	e.mu.Unlock()
+	if _, _, err := e.Fetch(context.Background(), rpmVirt("a"), "repodata/repomd.xml", ""); err == nil {
+		t.Fatal("a copy past maxStaleAge should not be served")
+	}
+	if len(e.stale) != 0 {
+		t.Fatal("an expired copy should be dropped")
+	}
+}
+
+func TestServeStaleToggleOffClearsCopy(t *testing.T) {
+	var down atomic.Bool
+	e, _, flag := staleEngineErr(true, func() (*RPMMember, error) {
+		if down.Load() {
+			return nil, errors.New("upstream down")
+		}
+		return rpmPkgMember("a", "1"), nil
+	})
+	fetchRepomd(t, e)
+	flag.Store(false)
+	fetchRepomd(t, e)
+	if len(e.stale) != 0 {
+		t.Fatal("turning serve_stale off should drop the copy")
+	}
+	flag.Store(true)
+	down.Store(true)
+	if _, _, err := e.Fetch(context.Background(), rpmVirt("a"), "repodata/repomd.xml", ""); err == nil {
+		t.Fatal("re-enabling serve_stale must not resurrect the old copy")
+	}
+}
+
+func staleRemote(_ context.Context, name string) (*models.Remote, error) {
+	return &models.Remote{Name: name, RepoType: models.RepoTypeRemote, ServeStale: true}, nil
+}
+
+func TestServeStaleDebAndAlpine(t *testing.T) {
+	var down atomic.Bool
+	e := fakeEngine(nil)
+	e.getRemote = staleRemote
+	e.debMember = func(_ context.Context, name, _ string, _ bool) (*DebMember, error) {
+		if down.Load() {
+			return nil, errors.New("upstream down")
+		}
+		return suiteDebMember(name, "1"), nil
+	}
+	e.apkMember = func(_ context.Context, name, _ string) (*AlpineMember, error) {
+		if down.Load() {
+			return nil, errors.New("upstream down")
+		}
+		return &AlpineMember{RemoteName: name, Index: []byte(apkRec("foo", "1-r0", "x86_64"))}, nil
+	}
+	for _, tc := range []struct {
+		virt models.Virtual
+		path string
+	}{
+		{debVirt("a"), "dists/s/Release"},
+		{alpineVirt("a"), "x86_64/APKINDEX.tar.gz"},
+	} {
+		down.Store(false)
+		first, _, err := e.Fetch(context.Background(), tc.virt, tc.path, "")
+		if err != nil {
+			t.Fatal(err)
+		}
+		down.Store(true)
+		got, _, err := e.Fetch(context.Background(), tc.virt, tc.path, "")
+		if err != nil || string(got) != string(first) {
+			t.Fatalf("%s: want the previous copy while the member is down, got err %v", tc.virt.PackageType, err)
+		}
+	}
+}
+
+func TestServeStaleEvictionDropsLateRefresh(t *testing.T) {
+	release := make(chan struct{})
+	var slow atomic.Bool
+	e := fakeEngine(nil)
+	e.mergeTTL = time.Minute
+	e.getRemote = staleRemote
+	e.debMember = func(_ context.Context, name, base string, _ bool) (*DebMember, error) {
+		if base == "dists/s0/" && slow.Load() {
+			<-release
+		}
+		return suiteDebMember(name, "1"), nil
+	}
+	fetch := func(suite string) {
+		t.Helper()
+		if _, _, err := e.Fetch(context.Background(), debVirt("a"), "dists/"+suite+"/Release", ""); err != nil {
+			t.Fatal(err)
+		}
+	}
+	fetch("s0")
+	slow.Store(true)
+	e.mu.Lock()
+	e.gens["deb\x00v\x00dists/s0/"].at = time.Time{}
+	e.mu.Unlock()
+	fetch("s0")
+	for i := range maxDebSuites {
+		time.Sleep(time.Microsecond)
+		fetch(fmt.Sprintf("s%d", i+1))
+	}
+	close(release)
+	sk := "deb\x00v\x00dists/s0/\x00a"
+	waitRefresh(e, sk)
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if e.gens["deb\x00v\x00dists/s0/"] != nil {
+		t.Fatal("suite s0 should be evicted")
+	}
+	if e.stale[sk] != nil {
+		t.Fatal("a refresh finishing after eviction must not re-insert its copy")
+	}
+	if len(e.stale) != maxDebSuites {
+		t.Fatalf("stale holds %d copies, want %d", len(e.stale), maxDebSuites)
+	}
+}
+
+func TestServeStaleEvictionDropsLateColdFetch(t *testing.T) {
+	release := make(chan struct{})
+	var slow atomic.Bool
+	e := fakeEngine(nil)
+	e.mergeTTL = time.Minute
+	e.getRemote = staleRemote
+	e.debMember = func(_ context.Context, name, base string, _ bool) (*DebMember, error) {
+		if base == "dists/s0/" && slow.Load() {
+			<-release
+		}
+		return suiteDebMember(name, "1"), nil
+	}
+	fetch := func(suite string) {
+		if _, _, err := e.Fetch(context.Background(), debVirt("a"), "dists/"+suite+"/Release", ""); err != nil {
+			t.Error(err)
+		}
+	}
+	sk := "deb\x00v\x00dists/s0/\x00a"
+	fetch("s0")
+	slow.Store(true)
+	e.mu.Lock()
+	e.gens["deb\x00v\x00dists/s0/"].at = time.Time{}
+	delete(e.stale, sk)
+	e.mu.Unlock()
+	done := make(chan struct{})
+	go func() { defer close(done); fetch("s0") }()
+	for {
+		e.mu.Lock()
+		cold := e.stale[sk] != nil
+		e.mu.Unlock()
+		if cold {
+			break
+		}
+		time.Sleep(time.Millisecond)
+	}
+	for i := range maxDebSuites {
+		time.Sleep(time.Microsecond)
+		fetch(fmt.Sprintf("s%d", i+1))
+	}
+	close(release)
+	<-done
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if e.stale[sk] != nil {
+		t.Fatal("a cold fetch finishing after eviction must not re-insert its copy")
+	}
+}
+
+func TestMergePrunesExpiredCopiesOfOtherMembers(t *testing.T) {
+	e, _ := staleEngine(false, func() *RPMMember { return rpmPkgMember("a", "1") })
+	e.stale = map[string]*staleEntry{"rpm\x00gone\x00x": {m: rpmPkgMember("x", "1"), at: time.Now().Add(-maxStaleAge)}}
+	fetchRepomd(t, e)
+	if len(e.stale) != 0 {
+		t.Fatal("any merge should prune expired copies of removed members")
+	}
+}
+
+func primary(m any) string { return string(m.(*RPMMember).Data["primary"]) }
+
+func TestServeStaleLookupErrorKeepsCopy(t *testing.T) {
+	release := make(chan struct{})
+	started := make(chan struct{}, 1)
+	var slow atomic.Bool
+	e, calls := staleEngine(true, func() *RPMMember {
+		if slow.Load() {
+			select {
+			case started <- struct{}{}:
+			default:
+			}
+			<-release
+			return rpmPkgMember("a", "2")
+		}
+		return rpmPkgMember("a", "1")
+	})
+	v1 := fetchRepomd(t, e)
+	slow.Store(true)
+	e.getRemote = func(context.Context, string) (*models.Remote, error) { return nil, errors.New("db down") }
+
+	done := make(chan string)
+	go func() { done <- fetchRepomd(t, e) }()
+	select {
+	case got := <-done:
+		if got != v1 {
+			t.Fatal("a failed remote lookup should serve the previous copy")
+		}
+	case <-time.After(time.Second):
+		t.Fatal("merge blocked on the member fetch instead of serving the copy")
+	}
+	select {
+	case <-started:
+	case <-time.After(time.Second):
+		t.Fatal("no background refresh started")
+	}
+	if n := calls.Load(); n != 2 {
+		t.Fatalf("member fetched %d times, want 2 (cold + one background refresh)", n)
+	}
+	close(release)
+	waitRefresh(e, rpmStaleKey)
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if se := e.stale[rpmStaleKey]; se == nil || se.m == nil || primary(se.m) != primary(rpmPkgMember("a", "2")) {
+		t.Fatal("the background refresh should replace the copy")
+	}
+}
+
+func TestServeStaleLookupErrorDuringColdFetch(t *testing.T) {
+	e := fakeEngine(nil)
+	e.getRemote = staleRemote
+	ctx := context.Background()
+	entered, release := make(chan struct{}), make(chan struct{})
+	first := make(chan *RPMMember)
+	go func() {
+		m, err := staleMember(ctx, e, "rpm\x00v", "a", func(context.Context) (*RPMMember, error) {
+			close(entered)
+			<-release
+			return rpmPkgMember("a", "1"), nil
+		})
+		if err != nil {
+			t.Error(err)
+		}
+		first <- m
+	}()
+	<-entered
+	e.mu.Lock()
+	placeholder := e.stale[rpmStaleKey]
+	e.mu.Unlock()
+	if placeholder == nil || placeholder.m != nil {
+		t.Fatal("a cold fetch should hold an empty placeholder")
+	}
+
+	e.getRemote = func(context.Context, string) (*models.Remote, error) { return nil, errors.New("db down") }
+	m, err := staleMember(ctx, e, "rpm\x00v", "a", func(context.Context) (*RPMMember, error) { return rpmPkgMember("a", "2"), nil })
+	if err != nil || primary(m) != primary(rpmPkgMember("a", "2")) {
+		t.Fatalf("a failed lookup during a cold fetch should fetch directly, got %v", err)
+	}
+	e.mu.Lock()
+	if e.stale[rpmStaleKey] != placeholder || placeholder.m != nil {
+		e.mu.Unlock()
+		t.Fatal("a failed lookup must not touch the in-flight placeholder")
+	}
+	e.mu.Unlock()
+
+	close(release)
+	if got := <-first; got == nil || primary(got) != primary(rpmPkgMember("a", "1")) {
+		t.Fatal("the cold fetch should return its own result")
+	}
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if se := e.stale[rpmStaleKey]; se == nil || se.m == nil || primary(se.m) != primary(rpmPkgMember("a", "1")) {
+		t.Fatal("the cold fetch should store its own result")
+	}
+}
+
+func TestServeStaleLookupErrorColdFetchStoresNothing(t *testing.T) {
+	e, calls := staleEngine(true, func() *RPMMember { return rpmPkgMember("a", "1") })
+	e.getRemote = func(context.Context, string) (*models.Remote, error) { return nil, errors.New("db down") }
+	fetchRepomd(t, e)
+	if calls.Load() != 1 {
+		t.Fatalf("cold member with a failed lookup should block on one fetch, got %d", calls.Load())
+	}
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if _, ok := e.stale[rpmStaleKey]; ok {
+		t.Fatal("a failed lookup must not create a copy")
+	}
+}
+
+func TestNonServeStaleLookupErrorStoresNothing(t *testing.T) {
+	e, calls := staleEngine(false, func() *RPMMember { return rpmPkgMember("a", "1") })
+	fetchRepomd(t, e)
+	e.getRemote = func(context.Context, string) (*models.Remote, error) { return nil, errors.New("db down") }
+	fetchRepomd(t, e)
+	if calls.Load() != 2 {
+		t.Fatalf("each merge should fetch the member, got %d", calls.Load())
+	}
+	e.mu.Lock()
+	defer e.mu.Unlock()
+	if _, ok := e.stale[rpmStaleKey]; ok {
+		t.Fatal("a failed lookup must not create a copy for a non-serve_stale member")
+	}
+}

internal/virtual/engine_test.go

@@ -19,7 +19,7 @@ func fakeEngine(members map[string]*RPMMember) *Engine {
 		getRemote: func(_ context.Context, name string) (*models.Remote, error) {
 			return &models.Remote{Name: name, RepoType: models.RepoTypeRemote}, nil
 		},
-		rpmMember: func(_ context.Context, name string) (*RPMMember, error) {
+		rpmMember: func(_ context.Context, name string, _ bool) (*RPMMember, error) {
 			if m := members[name]; m != nil {
 				return m, nil
 			}
@@ -75,7 +75,7 @@ func TestFetchRPMLocalMemberWithoutRepodataIsUpstreamError(t *testing.T) {
 	e := fakeEngine(map[string]*RPMMember{
 		"a": {RemoteName: "a", Data: map[string][]byte{"primary": primaryXML(primaryPkgXML("foo", "1", "aaa", "foo.rpm"))}},
 	})
-	e.rpmMember = func(_ context.Context, name string) (*RPMMember, error) {
+	e.rpmMember = func(_ context.Context, name string, _ bool) (*RPMMember, error) {
 		if name == "local" {
 			return nil, fmt.Errorf("local/repodata/repomd.xml: %w", ErrNotFound)
 		}
@@ -91,7 +91,7 @@ func TestFetchRPMMergeIsShared(t *testing.T) {
 	var calls atomic.Int32
 	e := fakeEngine(nil)
 	e.mergeTTL = time.Minute
-	e.rpmMember = func(_ context.Context, name string) (*RPMMember, error) {
+	e.rpmMember = func(_ context.Context, name string, _ bool) (*RPMMember, error) {
 		calls.Add(1)
 		time.Sleep(20 * time.Millisecond)
 		return &RPMMember{RemoteName: name, Data: map[string][]byte{"primary": primaryXML(primaryPkgXML("foo", "1", "aaa", "foo.rpm"))}}, nil
@@ -160,7 +160,7 @@ func TestMemberRedirect(t *testing.T) {
 
 func debEngine(members map[string]*DebMember) *Engine {
 	e := fakeEngine(nil)
-	e.debMember = func(_ context.Context, name, base string) (*DebMember, error) {
+	e.debMember = func(_ context.Context, name, base string, _ bool) (*DebMember, error) {
 		if m := members[name]; m != nil && base == "" {
 			return m, nil
 		}
@@ -213,7 +213,7 @@ func TestFetchDebMergeIsSharedPerSuite(t *testing.T) {
 	var calls atomic.Int32
 	e := fakeEngine(nil)
 	e.mergeTTL = time.Minute
-	e.debMember = func(_ context.Context, name, base string) (*DebMember, error) {
+	e.debMember = func(_ context.Context, name, base string, _ bool) (*DebMember, error) {
 		calls.Add(1)
 		m := flatDebMember(name, "", debStanza("foo", "1", "amd64", "pool/foo.deb"))
 		return &m, nil
@@ -307,7 +307,9 @@ func suiteDebMember(name, version string) *DebMember {
 func TestFetchDebByHashSurvivesRemerge(t *testing.T) {
 	version := "1"
 	e := fakeEngine(nil)
-	e.debMember = func(_ context.Context, name, _ string) (*DebMember, error) { return suiteDebMember(name, version), nil }
+	e.debMember = func(_ context.Context, name, _ string, _ bool) (*DebMember, error) {
+		return suiteDebMember(name, version), nil
+	}
 	virt := debVirt("a")
 	fetch := func(p string) []byte {
 		body, _, err := e.Fetch(context.Background(), virt, p, "")
@@ -340,7 +342,9 @@ func TestFetchDebByHashSurvivesRemerge(t *testing.T) {
 func TestFetchDebBoundsSuitesPerVirtual(t *testing.T) {
 	e := fakeEngine(nil)
 	e.mergeTTL = time.Minute
-	e.debMember = func(_ context.Context, name, _ string) (*DebMember, error) { return suiteDebMember(name, "1"), nil }
+	e.debMember = func(_ context.Context, name, _ string, _ bool) (*DebMember, error) {
+		return suiteDebMember(name, "1"), nil
+	}
 	other := models.Virtual{Name: "other", PackageType: models.PackageDeb, Members: []string{"a"}}
 	if _, _, err := e.Fetch(context.Background(), other, "dists/keep/Release", ""); err != nil {
 		t.Fatal(err)
@@ -367,3 +371,203 @@ func TestFetchDebBoundsSuitesPerVirtual(t *testing.T) {
 		t.Fatal("another virtual's suites must not be evicted")
 	}
 }
+
+func fetchFilesInput(n int) map[string]string {
+	files := map[string]string{}
+	for i := range n {
+		files[fmt.Sprintf("k%d", i)] = fmt.Sprintf("f%d", i)
+	}
+	return files
+}
+
+func TestFetchFilesParallelRunsConcurrently(t *testing.T) {
+	var inFlight, peak atomic.Int32
+	release := make(chan struct{})
+	go func() {
+		for peak.Load() < maxFileFetches {
+			time.Sleep(time.Millisecond)
+		}
+		close(release)
+	}()
+	got, err := fetchFiles(context.Background(), true, fetchFilesInput(6), func(_ context.Context, f string) ([]byte, error) {
+		n := inFlight.Add(1)
+		defer inFlight.Add(-1)
+		for p := peak.Load(); n > p && !peak.CompareAndSwap(p, n); p = peak.Load() {
+		}
+		select {
+		case <-release:
+		case <-time.After(5 * time.Second):
+			return nil, errors.New("fetches never overlapped")
+		}
+		return []byte(f), nil
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if peak.Load() != maxFileFetches {
+		t.Fatalf("peak concurrency %d, want %d", peak.Load(), maxFileFetches)
+	}
+	for k, f := range fetchFilesInput(6) {
+		if string(got[k]) != f {
+			t.Fatalf("%s = %q, want %q", k, got[k], f)
+		}
+	}
+}
+
+func TestFetchFilesSerialByDefault(t *testing.T) {
+	var inFlight atomic.Int32
+	_, err := fetchFiles(context.Background(), false, fetchFilesInput(4), func(_ context.Context, f string) ([]byte, error) {
+		defer inFlight.Add(-1)
+		if inFlight.Add(1) > 1 {
+			return nil, errors.New("concurrent fetch")
+		}
+		time.Sleep(time.Millisecond)
+		return []byte(f), nil
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestFetchFilesErrorFailsMember(t *testing.T) {
+	boom := errors.New("boom")
+	for _, parallel := range []bool{false, true} {
+		got, err := fetchFiles(context.Background(), parallel, fetchFilesInput(5), func(_ context.Context, f string) ([]byte, error) {
+			if f == "f2" {
+				return nil, boom
+			}
+			return []byte(f), nil
+		})
+		if !errors.Is(err, boom) || got != nil {
+			t.Fatalf("parallel=%v: got %v, %v; want boom and no files", parallel, got, err)
+		}
+	}
+}
+
+func TestMergeRPMPassesParallelFetchAndKeepsMemberOrder(t *testing.T) {
+	e := fakeEngine(nil)
+	var sawParallel atomic.Bool
+	e.rpmMember = func(_ context.Context, name string, parallel bool) (*RPMMember, error) {
+		sawParallel.Store(parallel)
+		if name == "a" {
+			time.Sleep(20 * time.Millisecond)
+		}
+		return &RPMMember{RemoteName: name, Data: map[string][]byte{"primary": primaryXML(primaryPkgXML("foo-"+name, "1", name+"sum", name+".rpm"))}}, nil
+	}
+	virt := rpmVirt("a", "b")
+	virt.ParallelFetch = true
+	repo, err := e.mergeRPM(context.Background(), virt)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !sawParallel.Load() {
+		t.Fatal("parallel_fetch not passed to member fetch")
+	}
+	primary := regexp.MustCompile(`repodata/[0-9a-f]+-primary\.xml\.gz`).FindString(string(repo.Repomd))
+	body := string(gunzip(t, repo.Files[primary]))
+	if a, b := strings.Index(body, "a.rpm"), strings.Index(body, "b.rpm"); a < 0 || b < a {
+		t.Fatalf("member priority order lost:\n%s", body)
+	}
+}
+
+func TestFetchFilesErrorCancelsRest(t *testing.T) {
+	boom := errors.New("boom")
+	var calls atomic.Int32
+	_, err := fetchFiles(context.Background(), true, fetchFilesInput(8), func(ctx context.Context, _ string) ([]byte, error) {
+		if calls.Add(1) == 1 {
+			return nil, boom
+		}
+		select {
+		case <-ctx.Done():
+			return nil, ctx.Err()
+		case <-time.After(5 * time.Second):
+			return nil, errors.New("in-flight fetch not cancelled")
+		}
+	})
+	if !errors.Is(err, boom) {
+		t.Fatalf("got %v, want boom", err)
+	}
+	if n := calls.Load(); n > maxFileFetches {
+		t.Fatalf("%d fetches started after the first error, want at most %d", n, maxFileFetches)
+	}
+}
+
+// overlapEngine serves fixed member index paths and records the peak number of
+// index file fetches in flight. Each fetch blocks until want are in flight (or
+// fails after 5s); with want 1 it also holds briefly so a stray overlap shows.
+func overlapEngine(index map[string][]byte, want int32, peak *atomic.Int32) *Engine {
+	e := fakeEngine(nil)
+	var inFlight atomic.Int32
+	e.memberPath = func(_ context.Context, _ models.Remote, path string) ([]byte, error) {
+		body, ok := index[path]
+		if !ok {
+			return nil, fmt.Errorf("%s: %w", path, ErrNotFound)
+		}
+		if strings.HasSuffix(path, "Release") || strings.HasSuffix(path, "repomd.xml") {
+			return body, nil
+		}
+		n := inFlight.Add(1)
+		defer inFlight.Add(-1)
+		for p := peak.Load(); n > p && !peak.CompareAndSwap(p, n); p = peak.Load() {
+		}
+		for deadline := time.Now().Add(5 * time.Second); peak.Load() < want; {
+			if time.Now().After(deadline) {
+				return nil, fmt.Errorf("%s: only %d of %d fetches in flight", path, peak.Load(), want)
+			}
+			time.Sleep(time.Millisecond)
+		}
+		if want == 1 {
+			time.Sleep(10 * time.Millisecond)
+		}
+		return body, nil
+	}
+	return e
+}
+
+func TestFetchRPMMemberHonoursParallelFetch(t *testing.T) {
+	index := map[string][]byte{
+		"repodata/repomd.xml": []byte(`<repomd>` +
+			`<data type="primary"><location href="repodata/p.xml"/></data>` +
+			`<data type="filelists"><location href="repodata/f.xml"/></data>` +
+			`<data type="other"><location href="repodata/o.xml"/></data></repomd>`),
+		"repodata/p.xml": []byte("P"), "repodata/f.xml": []byte("F"), "repodata/o.xml": []byte("O"),
+	}
+	for parallel, want := range map[bool]int32{false: 1, true: 3} {
+		var peak atomic.Int32
+		m, err := overlapEngine(index, want, &peak).fetchRPMMember(context.Background(), "a", parallel)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if peak.Load() != want {
+			t.Errorf("parallel=%v: peak concurrency %d, want %d", parallel, peak.Load(), want)
+		}
+		if string(m.Data["primary"]) != "P" || string(m.Data["filelists"]) != "F" || string(m.Data["other"]) != "O" {
+			t.Errorf("parallel=%v: data %q", parallel, m.Data)
+		}
+	}
+}
+
+func TestFetchDebMemberHonoursParallelFetch(t *testing.T) {
+	index := map[string][]byte{
+		"dists/s/Release": []byte("Suite: s\nSHA256:\n" +
+			" aa 1 main/binary-amd64/Packages\n" +
+			" bb 1 contrib/binary-amd64/Packages\n" +
+			" cc 1 non-free/binary-amd64/Packages\n"),
+		"dists/s/main/binary-amd64/Packages":     []byte("M"),
+		"dists/s/contrib/binary-amd64/Packages":  []byte("C"),
+		"dists/s/non-free/binary-amd64/Packages": []byte("N"),
+	}
+	for parallel, want := range map[bool]int32{false: 1, true: 3} {
+		var peak atomic.Int32
+		m, err := overlapEngine(index, want, &peak).fetchDebMember(context.Background(), "a", "dists/s/", parallel)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if peak.Load() != want {
+			t.Errorf("parallel=%v: peak concurrency %d, want %d", parallel, peak.Load(), want)
+		}
+		if string(m.Packages["main/binary-amd64/"]) != "M" || string(m.Packages["non-free/binary-amd64/"]) != "N" {
+			t.Errorf("parallel=%v: packages %q", parallel, m.Packages)
+		}
+	}
+}

internal/virtual/goproxy_merger.go

@@ -194,10 +194,18 @@ func (e *Engine) goList(ctx context.Context, virt models.Virtual, escMod string)
 	return nil, ErrNotFound
 }
 
-// goProbe returns the first member, in member order, that serves path.
+// goProbe returns the first member, in member order, that serves path, else
+// the first member quarantining it so the client sees that refusal.
 func (e *Engine) goProbe(ctx context.Context, virt models.Virtual, path string) (string, error) {
+	held := ""
 	for _, name := range virt.Members {
 		m, err := e.goMember(ctx, name, path)
+		if proxy.Quarantined(err) {
+			if held == "" {
+				held = name
+			}
+			continue
+		}
 		if err != nil {
 			return "", fmt.Errorf("virtual %q: member %q: %v", virt.Name, name, err)
 		}
@@ -205,6 +213,9 @@ func (e *Engine) goProbe(ctx context.Context, virt models.Virtual, path string)
 			return name, nil
 		}
 	}
+	if held != "" {
+		return held, nil
+	}
 	return "", ErrNotFound
 }
 

internal/virtual/goproxy_merger_test.go

@@ -3,11 +3,15 @@ package virtual
 import (
 	"context"
 	"errors"
+	"fmt"
+	"net/http"
+	"strings"
 	"sync"
 	"testing"
 	"time"
 
 	"git.unkin.net/unkin/artifactapi/internal/database"
+	"git.unkin.net/unkin/artifactapi/internal/proxy"
 	"git.unkin.net/unkin/artifactapi/internal/testsupport"
 	"git.unkin.net/unkin/artifactapi/pkg/models"
 )
@@ -181,3 +185,32 @@ func TestGoProxyLocalMemberNotFound(t *testing.T) {
 		t.Fatalf("module only absent from a local member must be not-found, got %v", err)
 	}
 }
+
+func TestGoProbeFallsThroughQuarantine(t *testing.T) {
+	quarantined := &proxy.ProxyError{Status: http.StatusForbidden, Message: "quarantined", Published: time.Now()}
+	served := map[string]bool{}
+	e := &Engine{
+		mergeTTL: time.Minute,
+		getRemote: func(_ context.Context, name string) (*models.Remote, error) {
+			return &models.Remote{Name: name, RepoType: models.RepoTypeRemote}, nil
+		},
+		goMember: func(_ context.Context, name, path string) (*GoMember, error) {
+			if name == "internal" && strings.HasSuffix(path, ".info") {
+				return nil, fmt.Errorf("fetch: %w", quarantined)
+			}
+			if served[name] {
+				return &GoMember{RemoteName: name, Body: []byte("{}")}, nil
+			}
+			return &GoMember{RemoteName: name}, nil
+		},
+	}
+	zip := "example.com/mod/@v/v0.0.0-20260101-abcdef.zip"
+	served["goproxy"] = true
+	if _, loc, err := e.GoProxy(context.Background(), goVirt(), zip, ""); err != nil || loc != "/api/v1/remote/goproxy/"+zip {
+		t.Errorf("next member serving: %q %v", loc, err)
+	}
+	served["goproxy"] = false
+	if _, loc, err := e.GoProxy(context.Background(), goVirt(), zip, ""); err != nil || loc != "/api/v1/remote/internal/"+zip {
+		t.Errorf("only quarantined member: %q %v, want redirect to its refusal", loc, err)
+	}
+}

migrations/0006_quarantine_days.sql

@@ -0,0 +1 @@
+UPDATE remotes SET quarantine_days = 3 WHERE quarantine_days IS NULL OR quarantine_days <= 0;

migrations/0007_virtual_parallel_fetch.sql

@@ -0,0 +1 @@
+ALTER TABLE virtuals ADD COLUMN IF NOT EXISTS parallel_fetch BOOLEAN NOT NULL DEFAULT FALSE;

migrations/0008_remote_serve_stale.sql

@@ -0,0 +1,3 @@
+-- serve_stale lets a virtual merge a member's previous index while that member
+-- refreshes in the background.
+ALTER TABLE remotes ADD COLUMN IF NOT EXISTS serve_stale BOOLEAN NOT NULL DEFAULT FALSE;

migrations/0009_remote_sync_interval.sql

@@ -0,0 +1 @@
+ALTER TABLE remotes ADD COLUMN IF NOT EXISTS sync_interval INTEGER NOT NULL DEFAULT 0;

migrations/0010_remote_cache_repodata.sql

@@ -0,0 +1 @@
+ALTER TABLE remotes ADD COLUMN IF NOT EXISTS cache_repodata BOOLEAN NOT NULL DEFAULT FALSE;

pkg/models/remote.go

@@ -2,6 +2,7 @@ package models
 
 import (
 	"fmt"
+	"hash/fnv"
 	"net/url"
 	"regexp"
 	"time"
@@ -55,6 +56,9 @@ type Remote struct {
 	ImmutableTTL int  `json:"immutable_ttl"`
 	MutableTTL   int  `json:"mutable_ttl"`
 	CheckMutable bool `json:"check_mutable"`
+	// SyncInterval (seconds) sets how often a github_* remote rescans releases
+	// and rebuilds its index; 0 falls back to mutable_ttl.
+	SyncInterval int `json:"sync_interval,omitempty"`
 
 	// Upstream HTTP timeouts in seconds. 0 means use the server default.
 	UpstreamDialTimeout           int `json:"upstream_dial_timeout,omitempty"`
@@ -73,10 +77,17 @@ type Remote struct {
 	QuarantineDays    int  `json:"quarantine_days,omitempty"`
 
 	StaleOnError bool `json:"stale_on_error"`
+	// ServeStale lets an rpm/deb/alpine virtual merge this member's previous
+	// index while it refreshes in the background.
+	ServeStale bool `json:"serve_stale"`
 
 	ReleasesRemote string `json:"releases_remote,omitempty"`
 	ManagedBy      string `json:"managed_by,omitempty"`
 
+	// CacheRepodata keeps a github_rpm remote's generated repodata in memory,
+	// rebuilding it only when its metadata rows change.
+	CacheRepodata bool `json:"cache_repodata"`
+
 	CreatedAt time.Time `json:"created_at"`
 	UpdatedAt time.Time `json:"updated_at"`
 }
@@ -149,6 +160,65 @@ func (r *Remote) ValidateMirrorStrategy() error {
 	return nil
 }
 
+// SyncPeriod is the background sync cadence: sync_interval, else mutable_ttl,
+// else def.
+func (r Remote) SyncPeriod(def time.Duration) time.Duration {
+	if r.SyncInterval > 0 {
+		return time.Duration(r.SyncInterval) * time.Second
+	}
+	if r.MutableTTL > 0 {
+		return time.Duration(r.MutableTTL) * time.Second
+	}
+	return def
+}
+
+// SyncSchedule is the sync_interval and the stable per-remote phase offsetting
+// its slots, so remotes sharing an interval spread across it; both zero when
+// sync_interval is unset.
+func (r Remote) SyncSchedule() (interval, phase time.Duration) {
+	if r.SyncInterval <= 0 {
+		return 0, 0
+	}
+	interval = time.Duration(r.SyncInterval) * time.Second
+	return interval, syncPhase(r.Name, interval)
+}
+
+func syncPhase(name string, period time.Duration) time.Duration {
+	h := fnv.New64a()
+	_, _ = h.Write([]byte(name))
+	return time.Duration(h.Sum64() % uint64(period))
+}
+
+// MaxSyncInterval caps sync_interval at 30 days (seconds).
+const MaxSyncInterval = 30 * 24 * 60 * 60
+
+// ValidateSyncInterval rejects a sync_interval outside 0..MaxSyncInterval, or
+// one set on a type without a background syncer.
+func (r *Remote) ValidateSyncInterval() error {
+	if r.SyncInterval < 0 || r.SyncInterval > MaxSyncInterval {
+		return fmt.Errorf("sync_interval must be between 0 and %d, got %d", MaxSyncInterval, r.SyncInterval)
+	}
+	switch r.PackageType {
+	case PackageGitHubRPM, PackageGitHubDeb, PackageGitHubAlpine:
+	default:
+		if r.SyncInterval > 0 {
+			return fmt.Errorf("sync_interval is only valid on github_rpm, github_deb and github_alpine remotes")
+		}
+	}
+	return nil
+}
+
+// DefaultQuarantineDays matches the remotes.quarantine_days column default.
+const DefaultQuarantineDays = 3
+
+// ValidateQuarantine rejects an enabled quarantine with no window.
+func (r *Remote) ValidateQuarantine() error {
+	if r.QuarantineEnabled && r.QuarantineDays < 1 {
+		return fmt.Errorf("quarantine_days must be at least 1 when quarantine_enabled is set")
+	}
+	return nil
+}
+
 // ValidatePatterns ensures every configured regex compiles. Storing an
 // invalid pattern would otherwise be silently dropped at match time, which
 // for the blocklist is a fail-open: a mistyped deny rule becomes a no-op.

pkg/models/remote_test.go

@@ -2,8 +2,10 @@ package models
 
 import (
 	"encoding/json"
+	"fmt"
 	"strings"
 	"testing"
+	"time"
 )
 
 func TestRemote_ValidatePatterns(t *testing.T) {
@@ -123,3 +125,68 @@ func TestValidateMirrorlist(t *testing.T) {
 		})
 	}
 }
+
+func TestSyncPeriod(t *testing.T) {
+	const def = 5 * time.Minute
+	for _, tc := range []struct {
+		interval, ttl int
+		want          time.Duration
+	}{
+		{0, 0, def},
+		{0, 600, 10 * time.Minute},
+		{3600, 600, time.Hour},
+	} {
+		r := Remote{SyncInterval: tc.interval, MutableTTL: tc.ttl}
+		if got := r.SyncPeriod(def); got != tc.want {
+			t.Errorf("SyncPeriod(interval=%d, ttl=%d) = %v, want %v", tc.interval, tc.ttl, got, tc.want)
+		}
+	}
+}
+
+func TestValidateSyncInterval(t *testing.T) {
+	for _, tc := range []struct {
+		pt       PackageType
+		interval int
+		ok       bool
+	}{
+		{PackageGitHubRPM, 3600, true},
+		{PackageGitHubDeb, 3600, true},
+		{PackageGitHubAlpine, 3600, true},
+		{PackageGitHubRPM, -1, false},
+		{PackageGitHubRPM, MaxSyncInterval, true},
+		{PackageGitHubRPM, MaxSyncInterval + 1, false},
+		{PackageRPM, 0, true},
+		{PackageRPM, 60, false},
+	} {
+		r := Remote{PackageType: tc.pt, SyncInterval: tc.interval}
+		if err := r.ValidateSyncInterval(); (err == nil) != tc.ok {
+			t.Errorf("%s interval=%d: err=%v, want ok=%v", tc.pt, tc.interval, err, tc.ok)
+		}
+	}
+}
+
+func TestSyncSchedule(t *testing.T) {
+	if i, p := (Remote{Name: "x", MutableTTL: 60}).SyncSchedule(); i != 0 || p != 0 {
+		t.Fatalf("unset sync_interval schedule = %v/%v, want 0/0", i, p)
+	}
+
+	const period = 5 * time.Minute
+	phases := map[time.Duration]bool{}
+	var lo, hi time.Duration = period, 0
+	for i := 0; i < 13; i++ {
+		r := Remote{Name: fmt.Sprintf("github-rpm-%02d", i), SyncInterval: int(period.Seconds())}
+		interval, p := r.SyncSchedule()
+		if interval != period || p < 0 || p >= period {
+			t.Fatalf("%s: schedule %v/%v outside [0, %v)", r.Name, interval, p, period)
+		}
+		if _, again := r.SyncSchedule(); again != p {
+			t.Fatalf("%s: phase not stable", r.Name)
+		}
+		p = p.Truncate(time.Second)
+		phases[p] = true
+		lo, hi = min(lo, p), max(hi, p)
+	}
+	if len(phases) != 13 || hi-lo < period/2 {
+		t.Fatalf("phases bunched: %d distinct, spread %v of %v", len(phases), hi-lo, period)
+	}
+}

pkg/models/virtual.go

@@ -3,11 +3,12 @@ package models
 import "time"
 
 type Virtual struct {
-	Name        string      `json:"name"`
+	Name          string      `json:"name"`
-	PackageType PackageType `json:"package_type"`
+	PackageType   PackageType `json:"package_type"`
-	Description string      `json:"description,omitempty"`
+	Description   string      `json:"description,omitempty"`
-	Members     []string    `json:"members"`
+	Members       []string    `json:"members"`
-	ManagedBy   string      `json:"managed_by,omitempty"`
+	ManagedBy     string      `json:"managed_by,omitempty"`
-	CreatedAt   time.Time   `json:"created_at"`
+	ParallelFetch bool        `json:"parallel_fetch"`
-	UpdatedAt   time.Time   `json:"updated_at"`
+	CreatedAt     time.Time   `json:"created_at"`
+	UpdatedAt     time.Time   `json:"updated_at"`
 }

ui/src/api/types.ts

@@ -8,6 +8,7 @@ export interface Remote {
   immutable_ttl: number;
   mutable_ttl: number;
   check_mutable: boolean;
+  sync_interval?: number;
   patterns: string[];
   blocklist: string[];
   mutable_patterns: string[];
@@ -17,6 +18,8 @@ export interface Remote {
   quarantine_enabled: boolean;
   quarantine_days: number;
   stale_on_error: boolean;
+  serve_stale: boolean;
+  cache_repodata: boolean;
   releases_remote: string;
   managed_by: string;
   created_at: string;
@@ -29,6 +32,7 @@ export interface Virtual {
   description: string;
   members: string[];
   managed_by: string;
+  parallel_fetch: boolean;
   created_at: string;
   updated_at: string;
 }

ui/src/pages/RemoteDetail.tsx

@@ -50,6 +50,10 @@ export function RemoteDetail() {
             <dd>{remote.check_mutable ? 'enabled' : 'disabled'}</dd>
             <dt>Stale on Error</dt>
             <dd>{remote.stale_on_error ? 'enabled' : 'disabled'}</dd>
+            <dt>Serve Stale</dt>
+            <dd>{remote.serve_stale ? 'enabled' : 'disabled'}</dd>
+            <dt>Cache Repodata</dt>
+            <dd>{remote.cache_repodata ? 'enabled' : 'disabled'}</dd>
             {remote.releases_remote && (
               <>
                 <dt>Releases Remote</dt>

ui/src/pages/Virtuals.tsx

@@ -26,6 +26,8 @@ export function Virtuals() {
       .finally(() => setLoading(false));
   }, []);
 
+  const selected = virtuals.find(x => x.name === expanded);
+
   function memberLink(name: string) {
     const remote = remoteMap[name];
     if (remote?.repo_type === 'local') {
@@ -67,6 +69,13 @@ export function Virtuals() {
               header: 'Description',
               render: (v: Virtual) => v.description || <span className="text-muted">—</span>,
             },
+            {
+              key: 'parallel',
+              header: 'Parallel',
+              render: (v: Virtual) =>
+                v.parallel_fetch ? <Badge variant="yellow">parallel</Badge> : <span className="text-muted">—</span>,
+              width: '100px',
+            },
             {
               key: 'managed',
               header: 'Managed',
@@ -81,30 +90,23 @@ export function Virtuals() {
         />
       )}
 
-      {expanded && (
+      {selected && (
         <div className="virtual-detail-panel">
-          <h3 className="section-label">Members of {expanded}</h3>
+          <h3 className="section-label">Members of {selected.name}</h3>
           <ul className="member-list">
-            {virtuals
+            {selected.members?.map((m, i) => {
-              .find(v => v.name === expanded)
+              const remote = remoteMap[m];
-              ?.members?.map((m, i) => {
+              const typeLabel = remote?.repo_type === 'local' ? 'local' : 'remote';
-                const remote = remoteMap[m];
+              return (
-                const typeLabel = remote?.repo_type === 'local' ? 'local' : 'remote';
+                <li key={m}>
-                return (
+                  <span className="member-priority">{i + 1}</span>
-                  <li key={m}>
+                  <Link to={memberLink(m)} className="mono">{m}</Link>
-                    <span className="member-priority">{i + 1}</span>
+                  <Badge variant={typeLabel === 'local' ? 'yellow' : 'default'}>{typeLabel}</Badge>
-                    <Link to={memberLink(m)} className="mono">{m}</Link>
+                </li>
-                    <Badge variant={typeLabel === 'local' ? 'yellow' : 'default'}>{typeLabel}</Badge>
+              );
-                  </li>
+            })}
-                );
-              })}
           </ul>
-          {(() => {
+          <UsageInstructions packageType={selected.package_type} repoClass="virtual" name={selected.name} defaultOpen />
-            const v = virtuals.find(x => x.name === expanded);
-            return v ? (
-              <UsageInstructions packageType={v.package_type} repoClass="virtual" name={v.name} defaultOpen />
-            ) : null;
-          })()}
         </div>
       )}
     </div>