feat: server-level GitHub machine credential for authenticated requests

Anonymous GitHub is capped at 60 requests/hour and cannot read private
repositories, so a machine credential usable by a free (non-enterprise)
account is needed to lift the request budget and reach private release
assets.

- Add internal/githubauth: a process-wide credential delivered via env/secret,
  applied by default to every outbound GitHub request.
- Support two modes: a Personal Access Token sent as `Authorization: Bearer`,
  and a GitHub App that mints a short-lived RS256 JWT (stdlib crypto, no new
  dependency), exchanges it for a ~1h installation token, caches it, and
  single-flights a refresh a few minutes before expiry.
- Inject the credential at the two GitHub call paths: the rpm github provider
  (releases scan + ranged asset-header GETs) and the generic byte proxy
  (private release-asset downloads for github.com hosts).
- Honor precedence: a remote's own username/password overrides the server
  credential; no credential configured stays anonymous.
- Fail closed at startup on partial App configuration; never persist the
  credential to the DB, return it from an API, or log it.
- Read GITHUB_TOKEN / GITHUB_APP_ID / GITHUB_APP_INSTALLATION_ID /
  GITHUB_APP_PRIVATE_KEY[_PATH] via the existing getenv convention.
- Document PAT vs App setup, the free-account fine-grained PAT scopes
  (Contents:read + Metadata:read), precedence, and the rate-limit implication.

README.md

@@ -46,12 +46,43 @@ metadata comes from a **ranged GET of just the RPM header** (the header sits at
 the front of the file, so the whole package is never downloaded); the sha256
 checksum comes from the GitHub asset `digest` when present, else a one-time
 lazy stream. Derived metadata is cached (keyed by asset) so repodata generation
-is cheap on repeat, and refreshed no more often than `mutable_ttl`.
+is served from primed DB rows, never a cold on-demand derive.
 
 Each package's `<location>` points back at the remote, which **302-redirects**
 the download to the `releases_remote` — an existing generic `github.com` remote
 that streams the actual bytes. `dnf` follows the redirect transparently.
 
+#### Background syncer
+
+A single process-wide **background syncer** keeps every `github_rpm` remote's
+derived metadata current off the client request path:
+
+- **Prime on create.** Creating a `github_rpm` remote enqueues a background prime
+  scan, so its metadata is derived right away without blocking the create call.
+  The first `dnf` request is served from cache. If a request arrives before the
+  prime lands, it returns a retryable `503` (with `Retry-After`) rather than
+  serving an empty repo or blocking on a multi-minute derive.
+- **Periodic re-check, driven by `mutable_ttl`.** Each remote is re-checked for
+  new or changed releases no more often than its `mutable_ttl`. New/changed
+  assets are derived incrementally; assets already cached are never re-fetched,
+  and assets that disappear upstream are pruned.
+- **ETag / 304 conditional requests.** The releases-list `ETag` is stored per
+  remote and sent as `If-None-Match`; a `304 Not Modified` means nothing changed
+  and the syncer derives nothing. GitHub does not count `304` conditional
+  responses against the rate limit, so an unchanged repo is nearly free — this is
+  the main lever keeping GitHub traffic low.
+- **Global rate limit.** Every GitHub call (releases list + each ranged asset
+  header GET) passes through a single token-bucket limiter **shared across all
+  remotes**, so GitHub is never hammered. Configure a token (`password`) on the
+  remote for the higher authenticated rate limit (~5000/hr vs ~60/hr
+  unauthenticated).
+- **Multi-replica coordination.** State is shared through the database. Before a
+  periodic scan a replica must atomically claim a per-remote lease
+  (`github_rpm_sync_state`: `last_synced_at`, `etag`, `sync_lease_owner`,
+  `sync_lease_expires`); only the winner scans. This bounds total GitHub load to
+  ~once per `mutable_ttl` regardless of replica count, and the shared `etag`
+  lets any replica issue the conditional request.
+
 ```hcl
 # Backend that serves the actual .rpm bytes from github.com.
 resource "artifactapi_remote_generic" "github" {
@@ -80,6 +111,71 @@ resource "artifactapi_remote_github_rpm" "acme-tools" {
 The repo is multi-arch (no `$basearch` needed) — `dnf` selects matching packages
 from the synthesized metadata.
 
+### GitHub authentication
+
+Anonymous GitHub is capped at **60 requests/hour** and cannot read private
+repositories. Configure a **server-level GitHub credential** to raise the ceiling
+to roughly **5000 requests/hour** and to read private-repo release assets. The
+credential is a process-wide machine identity applied by default to *every*
+outbound GitHub request — the releases scan, the ranged asset-header fetches, and
+the generic-github byte proxy that streams private release assets.
+
+The credential is read from the environment (deliver it from a Vault or
+Kubernetes secret). It is **never** stored per-remote in the database, **never**
+returned by any API, and **never** logged. Configure **exactly one** mode.
+
+**Precedence.** A remote's own `username`/`password` credential still wins for
+that remote's requests; the server credential is the default for everything else.
+With no credential configured at all, requests stay anonymous (current behavior).
+Partial configuration (e.g. an App id with no private key) is a **startup error**
+— artifactapi fails closed rather than silently falling back to anonymous.
+
+Both modes share the syncer's single global rate limiter, so a token simply
+raises the effective GitHub ceiling; the default limiter settings stay safe.
+
+#### Mode 1 — Personal Access Token (minimum viable, recommended for free accounts)
+
+Set `GITHUB_TOKEN`. It is sent as `Authorization: Bearer <token>`.
+
+Recommended free-account setup — a **fine-grained PAT** scoped to just the target
+repositories:
+
+1. GitHub → *Settings → Developer settings → Personal access tokens →
+   Fine-grained tokens → Generate new token*.
+2. Limit *Repository access* to the specific repo(s) serving releases.
+3. Grant repository permissions **Contents: Read-only** and **Metadata:
+   Read-only** (Metadata is mandatory and auto-selected).
+
+A classic PAT with the `repo` scope also works but is broader than necessary.
+
+```bash
+GITHUB_TOKEN=github_pat_xxxxxxxx
+```
+
+#### Mode 2 — GitHub App installation token (proper machine identity)
+
+A GitHub App is not tied to a personal account and can be created and installed on
+free personal repos. artifactapi mints a short-lived RS256 **JWT** from the app
+private key, exchanges it at `POST /app/installations/{id}/access_tokens` for a
+~1-hour **installation access token**, caches that token, and refreshes it a few
+minutes before expiry (thread-safe, single-flighted).
+
+1. GitHub → *Settings → Developer settings → GitHub Apps → New GitHub App*.
+2. Under *Permissions → Repository permissions* grant **Contents: Read-only**
+   (Metadata: Read-only is implied).
+3. Generate a **private key** (downloads a PEM) and note the **App ID**.
+4. *Install* the App on the account and select the target repositories, then read
+   the **Installation ID** from the installation URL
+   (`.../settings/installations/<installation-id>`).
+
+```bash
+GITHUB_APP_ID=123456
+GITHUB_APP_INSTALLATION_ID=7654321
+GITHUB_APP_PRIVATE_KEY_PATH=/etc/artifactapi/github-app.pem
+# or inline PEM (e.g. mounted from a secret):
+# GITHUB_APP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----"
+```
+
 ## Terraform
 
 Remotes and virtuals are managed by Terraform. Each package type has its own resource:
@@ -242,6 +338,15 @@ S3 client supports MinIO, Ceph RGW, and AWS S3 (via minio-go).
 | `MINIO_BUCKET` | `artifacts` | S3 bucket |
 | `MINIO_SECURE` | `false` | Use HTTPS for S3 |
 | `MINIO_REGION` | | S3 region (AWS) |
+| `GITHUB_SYNC_RATE` | `1` | `github_rpm` syncer global GitHub request rate (req/s), shared across all remotes. `1`/s = 3600/hr, under an authenticated token's ~5000/hr; unauthenticated (~60/hr) relies on ETag/304 |
+| `GITHUB_SYNC_BURST` | `5` | Token-bucket burst for the shared limiter |
+| `GITHUB_SYNC_WORKERS` | `3` | Concurrent `github_rpm` scan workers |
+| `GITHUB_SYNC_POLL_INTERVAL` | `60` | Base scheduler tick in seconds; per-remote cadence is its `mutable_ttl`, enforced by the DB lease |
+| `GITHUB_TOKEN` | | Server-level GitHub PAT (fine-grained or classic), sent as `Authorization: Bearer`. Applies to every GitHub request; per-remote creds override it. See [GitHub authentication](#github-authentication) |
+| `GITHUB_APP_ID` | | GitHub App id (App auth mode; mutually exclusive with `GITHUB_TOKEN`) |
+| `GITHUB_APP_INSTALLATION_ID` | | GitHub App installation id |
+| `GITHUB_APP_PRIVATE_KEY` | | GitHub App private key, inline PEM |
+| `GITHUB_APP_PRIVATE_KEY_PATH` | | GitHub App private key, file path (alternative to inline PEM) |
 
 ## Development
 

go.mod

@@ -15,6 +15,7 @@ require (
 	github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0
 	github.com/testcontainers/testcontainers-go/modules/redis v0.42.0
 	golang.org/x/crypto v0.51.0
+	golang.org/x/time v0.15.0
 	gopkg.in/yaml.v3 v3.0.1
 )
 

go.sum

@@ -234,6 +234,8 @@ golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
 golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
 golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
 golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
+golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
+golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=

internal/api/v2/errorpaths_test.go

@@ -57,7 +57,7 @@ func do(t *testing.T, h http.Handler, method, path, body string) int {
 }
 
 func TestRemotesErrorPaths(t *testing.T) {
-	h := NewRemotesHandler(closedDB(t)).Routes()
+	h := NewRemotesHandler(closedDB(t), nil).Routes()
 	if c := do(t, h, "GET", "/", ""); c != 500 {
 		t.Errorf("list with dead db = %d, want 500", c)
 	}

internal/api/v2/remotes.go

@@ -11,12 +11,19 @@ import (
 	"git.unkin.net/unkin/artifactapi/pkg/models"
 )
 
-type RemotesHandler struct {
-	db *database.DB
+// Primer enqueues a background metadata prime for a newly created remote so the
+// create call never blocks on a derive. *rpm.Syncer satisfies it.
+type Primer interface {
+	EnqueuePrime(remote models.Remote)
 }
 
-func NewRemotesHandler(db *database.DB) *RemotesHandler {
-	return &RemotesHandler{db: db}
+type RemotesHandler struct {
+	db     *database.DB
+	primer Primer
+}
+
+func NewRemotesHandler(db *database.DB, primer Primer) *RemotesHandler {
+	return &RemotesHandler{db: db, primer: primer}
 }
 
 func (h *RemotesHandler) Routes() chi.Router {
@@ -77,6 +84,11 @@ func (h *RemotesHandler) create(w http.ResponseWriter, r *http.Request) {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
 		return
 	}
+	// Prime a github_rpm remote's metadata in the background so its first
+	// repodata request is served from cache instead of a cold on-demand derive.
+	if h.primer != nil && remote.PackageType == models.PackageGitHubRPM {
+		h.primer.EnqueuePrime(remote)
+	}
 	writeJSON(w, http.StatusCreated, remote)
 }
 

internal/config/env.go

@@ -32,6 +32,30 @@ type Config struct {
 	TFSigningKeyPath       string
 	TFSigningKeyPassphrase string
 	TFProviderProtocols    string
+
+	// github_rpm background syncer. The syncer keeps derived RPM metadata for
+	// every github_rpm remote fresh off the client request path, sharing a
+	// single global token-bucket limiter across all remotes so GitHub is never
+	// hammered. Defaults are conservative: 1 req/s (3600/hr) sits well under an
+	// authenticated token's 5000/hr. Unauthenticated remotes (60/hr) lean on
+	// ETag/304 — an unchanged repo costs nothing — so keep those repos small or
+	// configure a token.
+	GitHubSyncRatePerSec   float64
+	GitHubSyncBurst        int
+	GitHubSyncWorkers      int
+	GitHubSyncPollInterval int
+
+	// Server-level GitHub machine credential, applied by default to every
+	// outbound GitHub request (releases scan, ranged asset fetches, and the
+	// generic-github byte proxy for private assets). Delivered via env/secret
+	// only — never stored per-remote, never returned by an API, never logged.
+	// Configure exactly one mode: a Personal Access Token, or a GitHub App
+	// (id + installation id + private key). Partial App config fails at startup.
+	GitHubToken             string
+	GitHubAppID             string
+	GitHubAppInstallationID string
+	GitHubAppPrivateKey     string
+	GitHubAppPrivateKeyPath string
 }
 
 func (c *Config) DatabaseDSN() string {
@@ -49,6 +73,23 @@ func Load() (*Config, error) {
 
 	s3Secure, _ := strconv.ParseBool(getenv("MINIO_SECURE", "false"))
 
+	syncRate, err := strconv.ParseFloat(getenv("GITHUB_SYNC_RATE", "1"), 64)
+	if err != nil {
+		return nil, fmt.Errorf("invalid GITHUB_SYNC_RATE: %w", err)
+	}
+	syncBurst, err := strconv.Atoi(getenv("GITHUB_SYNC_BURST", "5"))
+	if err != nil {
+		return nil, fmt.Errorf("invalid GITHUB_SYNC_BURST: %w", err)
+	}
+	syncWorkers, err := strconv.Atoi(getenv("GITHUB_SYNC_WORKERS", "3"))
+	if err != nil {
+		return nil, fmt.Errorf("invalid GITHUB_SYNC_WORKERS: %w", err)
+	}
+	syncPoll, err := strconv.Atoi(getenv("GITHUB_SYNC_POLL_INTERVAL", "60"))
+	if err != nil {
+		return nil, fmt.Errorf("invalid GITHUB_SYNC_POLL_INTERVAL: %w", err)
+	}
+
 	cfg := &Config{
 		ListenAddr: getenv("LISTEN_ADDR", ":8000"),
 
@@ -71,6 +112,17 @@ func Load() (*Config, error) {
 		TFSigningKeyPath:       getenv("TF_SIGNING_KEY_PATH", ""),
 		TFSigningKeyPassphrase: getenv("TF_SIGNING_KEY_PASSPHRASE", ""),
 		TFProviderProtocols:    getenv("TF_PROVIDER_PROTOCOLS", "5.0,6.0"),
+
+		GitHubSyncRatePerSec:   syncRate,
+		GitHubSyncBurst:        syncBurst,
+		GitHubSyncWorkers:      syncWorkers,
+		GitHubSyncPollInterval: syncPoll,
+
+		GitHubToken:             getenv("GITHUB_TOKEN", ""),
+		GitHubAppID:             getenv("GITHUB_APP_ID", ""),
+		GitHubAppInstallationID: getenv("GITHUB_APP_INSTALLATION_ID", ""),
+		GitHubAppPrivateKey:     getenv("GITHUB_APP_PRIVATE_KEY", ""),
+		GitHubAppPrivateKeyPath: getenv("GITHUB_APP_PRIVATE_KEY_PATH", ""),
 	}
 
 	return cfg, nil

internal/database/github_sync.go

@@ -0,0 +1,73 @@
+package database
+
+import (
+	"context"
+	"errors"
+	"time"
+
+	"github.com/jackc/pgx/v5"
+
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+// ListGitHubRPMRemotes returns every github_rpm remote so the syncer can sweep
+// them on each poll tick.
+func (db *DB) ListGitHubRPMRemotes(ctx context.Context) ([]models.Remote, error) {
+	rows, err := db.Pool.Query(ctx, `SELECT `+remoteCols+` FROM remotes WHERE package_type = $1 ORDER BY name`, models.PackageGitHubRPM)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	var remotes []models.Remote
+	for rows.Next() {
+		var r models.Remote
+		if err := scanRemote(rows, &r); err != nil {
+			return nil, err
+		}
+		remotes = append(remotes, r)
+	}
+	return remotes, rows.Err()
+}
+
+// ClaimGitHubSyncLease atomically claims the per-remote sync lease. It succeeds
+// (claimed=true) only when the remote is due — never synced, or synced longer
+// than freshness ago — and no live lease is held by another replica. This bounds
+// total GitHub load to roughly one scan per freshness window regardless of how
+// many replicas poll. The returned etag is the stored releases-list ETag, shared
+// across replicas so a conditional request can short-circuit an unchanged repo.
+// A zero freshness (used for prime scans) ignores the recency gate and claims
+// whenever no live lease is held.
+func (db *DB) ClaimGitHubSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (bool, string, error) {
+	row := db.Pool.QueryRow(ctx, `
+		INSERT INTO github_rpm_sync_state AS s (remote_name, sync_lease_owner, sync_lease_expires)
+		VALUES ($1, $2, now() + make_interval(secs => $4))
+		ON CONFLICT (remote_name) DO UPDATE
+			SET sync_lease_owner   = $2,
+			    sync_lease_expires = now() + make_interval(secs => $4)
+			WHERE (s.last_synced_at IS NULL OR s.last_synced_at < now() - make_interval(secs => $3))
+			  AND (s.sync_lease_expires IS NULL OR s.sync_lease_expires < now())
+		RETURNING s.etag
+	`, remoteName, owner, freshness.Seconds(), lease.Seconds())
+
+	var etag string
+	if err := row.Scan(&etag); err != nil {
+		if errors.Is(err, pgx.ErrNoRows) {
+			return false, "", nil
+		}
+		return false, "", err
+	}
+	return true, etag, nil
+}
+
+// ReleaseGitHubSyncLease records the completed scan and frees the lease. Only the
+// owning replica may release; last_synced_at advances so the next poll waits a
+// full freshness window, and etag is persisted for the next conditional request.
+func (db *DB) ReleaseGitHubSyncLease(ctx context.Context, remoteName, owner, etag string, syncedAt time.Time) error {
+	_, err := db.Pool.Exec(ctx, `
+		UPDATE github_rpm_sync_state
+		   SET last_synced_at = $3, etag = $4, sync_lease_owner = '', sync_lease_expires = NULL
+		 WHERE remote_name = $1 AND sync_lease_owner = $2
+	`, remoteName, owner, syncedAt, etag)
+	return err
+}

internal/database/github_sync_test.go

@@ -0,0 +1,95 @@
+package database
+
+import (
+	"testing"
+	"time"
+
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+func seedGitHubRPMRemote(t *testing.T, name string) {
+	t.Helper()
+	if err := testDB.CreateRemote(ctx(), &models.Remote{
+		Name: name, PackageType: models.PackageGitHubRPM, RepoType: models.RepoTypeRemote,
+		BaseURL: "https://api.github.com/repos/acme/tools", ReleasesRemote: "github", MutableTTL: 3600,
+	}); err != nil {
+		t.Fatalf("seed github_rpm remote: %v", err)
+	}
+}
+
+// TestGitHubSyncLease exercises the real SQL: exactly one replica may hold the
+// lease, the recency window blocks a too-soon periodic re-claim, and a prime
+// (freshness 0) bypasses recency but still respects a live lease.
+func TestGitHubSyncLease(t *testing.T) {
+	requireDB(t)
+	name := "gh-lease-" + time.Now().Format("150405.000000")
+	seedGitHubRPMRemote(t, name)
+
+	const lease = 15 * time.Minute
+	freshness := time.Hour
+
+	// First claim on a never-synced remote wins; etag starts empty.
+	claimed, etag, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-1", freshness, lease)
+	if err != nil || !claimed {
+		t.Fatalf("replica-1 first claim: claimed=%v err=%v", claimed, err)
+	}
+	if etag != "" {
+		t.Fatalf("initial etag should be empty, got %q", etag)
+	}
+
+	// A second replica cannot claim while the lease is held.
+	claimed2, _, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", freshness, lease)
+	if err != nil {
+		t.Fatalf("replica-2 claim err: %v", err)
+	}
+	if claimed2 {
+		t.Fatal("replica-2 claimed while replica-1 holds the lease")
+	}
+
+	// Replica 1 finishes: record the sync and persist an etag.
+	if err := testDB.ReleaseGitHubSyncLease(ctx(), name, "replica-1", `"etag-1"`, time.Now()); err != nil {
+		t.Fatalf("release: %v", err)
+	}
+
+	// A periodic re-claim inside the freshness window is blocked by recency.
+	claimed3, _, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", freshness, lease)
+	if err != nil {
+		t.Fatalf("replica-2 recency claim err: %v", err)
+	}
+	if claimed3 {
+		t.Fatal("periodic claim succeeded inside the freshness window")
+	}
+
+	// A prime (freshness 0) bypasses recency and reads the persisted etag.
+	claimed4, etag4, err := testDB.ClaimGitHubSyncLease(ctx(), name, "replica-2", 0, lease)
+	if err != nil || !claimed4 {
+		t.Fatalf("prime claim: claimed=%v err=%v", claimed4, err)
+	}
+	if etag4 != `"etag-1"` {
+		t.Fatalf("prime claim etag = %q, want persisted \"etag-1\"", etag4)
+	}
+}
+
+func TestListGitHubRPMRemotes(t *testing.T) {
+	requireDB(t)
+	name := "gh-list-" + time.Now().Format("150405.000000")
+	seedGitHubRPMRemote(t, name)
+	seedRemote(t, "generic-"+time.Now().Format("150405.000000"))
+
+	remotes, err := testDB.ListGitHubRPMRemotes(ctx())
+	if err != nil {
+		t.Fatalf("list: %v", err)
+	}
+	found := false
+	for _, r := range remotes {
+		if r.PackageType != models.PackageGitHubRPM {
+			t.Fatalf("non-github_rpm remote returned: %s (%s)", r.Name, r.PackageType)
+		}
+		if r.Name == name {
+			found = true
+		}
+	}
+	if !found {
+		t.Fatalf("seeded remote %q not returned", name)
+	}
+}

internal/database/postgres.go

@@ -164,6 +164,14 @@ func (db *DB) migrate() error {
 		ALTER TABLE rpm_metadata ADD COLUMN IF NOT EXISTS conflicts JSONB DEFAULT '[]';
 		ALTER TABLE rpm_metadata ADD COLUMN IF NOT EXISTS obsoletes JSONB DEFAULT '[]';
 
+		CREATE TABLE IF NOT EXISTS github_rpm_sync_state (
+			remote_name        TEXT PRIMARY KEY,
+			etag               TEXT DEFAULT '',
+			last_synced_at     TIMESTAMPTZ,
+			sync_lease_owner   TEXT DEFAULT '',
+			sync_lease_expires TIMESTAMPTZ
+		);
+
 		CREATE TABLE IF NOT EXISTS signing_keys (
 			purpose             TEXT PRIMARY KEY,
 			private_key_armor   TEXT NOT NULL,

internal/githubauth/app.go

@@ -0,0 +1,199 @@
+package githubauth
+
+import (
+	"context"
+	"crypto"
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/sha256"
+	"crypto/x509"
+	"encoding/base64"
+	"encoding/json"
+	"encoding/pem"
+	"errors"
+	"fmt"
+	"io"
+	"net/http"
+	"strings"
+	"sync"
+	"time"
+)
+
+const (
+	defaultAPIBase = "https://api.github.com"
+
+	// jwtLifetime is how long the app JWT is valid. GitHub caps it at 10 minutes;
+	// 9 leaves headroom for clock skew.
+	jwtLifetime = 9 * time.Minute
+	// jwtBackdate backdates iat to tolerate the app server's clock running behind
+	// GitHub's, which otherwise rejects the JWT.
+	jwtBackdate = 60 * time.Second
+	// refreshSkew refreshes the installation token this long before it expires so
+	// a request never races an expiry.
+	refreshSkew = 5 * time.Minute
+)
+
+type httpDoer interface {
+	Do(*http.Request) (*http.Response, error)
+}
+
+// appCredential mints installation access tokens for a GitHub App. It signs a
+// short-lived RS256 JWT with the app private key, exchanges it for a ~1h
+// installation token, caches that token, and refreshes it shortly before expiry.
+// Refreshes are single-flighted by holding the mutex across the exchange, so
+// concurrent callers coalesce onto one HTTP request and reuse the cached token.
+type appCredential struct {
+	appID          string
+	installationID string
+	key            *rsa.PrivateKey
+	apiBase        string
+	client         httpDoer
+
+	mu     sync.Mutex
+	token  string
+	expiry time.Time
+}
+
+func newAppCredential(opts Options) (*appCredential, error) {
+	if opts.AppID == "" {
+		return nil, errors.New("github app: GITHUB_APP_ID is required")
+	}
+	if opts.InstallationID == "" {
+		return nil, errors.New("github app: GITHUB_APP_INSTALLATION_ID is required")
+	}
+	pemBytes, err := loadPrivateKeyPEM(opts)
+	if err != nil {
+		return nil, err
+	}
+	key, err := parseRSAPrivateKey(pemBytes)
+	if err != nil {
+		return nil, err
+	}
+
+	apiBase := opts.apiBaseURL
+	if apiBase == "" {
+		apiBase = defaultAPIBase
+	}
+	client := opts.httpClient
+	if client == nil {
+		client = &http.Client{Timeout: 30 * time.Second}
+	}
+
+	return &appCredential{
+		appID:          opts.AppID,
+		installationID: opts.InstallationID,
+		key:            key,
+		apiBase:        strings.TrimRight(apiBase, "/"),
+		client:         client,
+	}, nil
+}
+
+// Token returns a cached installation token, refreshing it under a single-flight
+// lock when it is missing or within refreshSkew of expiry.
+func (a *appCredential) Token(ctx context.Context) (string, error) {
+	a.mu.Lock()
+	defer a.mu.Unlock()
+	if a.token != "" && time.Now().Before(a.expiry.Add(-refreshSkew)) {
+		return a.token, nil
+	}
+	if err := a.refreshLocked(ctx); err != nil {
+		return "", err
+	}
+	return a.token, nil
+}
+
+func (a *appCredential) refreshLocked(ctx context.Context) error {
+	jwt, err := mintJWT(a.appID, a.key, time.Now())
+	if err != nil {
+		return fmt.Errorf("github app: mint jwt: %w", err)
+	}
+
+	u := fmt.Sprintf("%s/app/installations/%s/access_tokens", a.apiBase, a.installationID)
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, nil)
+	if err != nil {
+		return err
+	}
+	req.Header.Set("Authorization", "Bearer "+jwt)
+	req.Header.Set("Accept", "application/vnd.github+json")
+	req.Header.Set("X-GitHub-Api-Version", "2022-11-28")
+
+	resp, err := a.client.Do(req)
+	if err != nil {
+		return fmt.Errorf("github app: token exchange: %w", err)
+	}
+	defer resp.Body.Close()
+	body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
+	if resp.StatusCode != http.StatusCreated && resp.StatusCode != http.StatusOK {
+		// Never echo the body verbatim — it can contain sensitive material.
+		return fmt.Errorf("github app: token exchange status %d", resp.StatusCode)
+	}
+
+	var out struct {
+		Token     string    `json:"token"`
+		ExpiresAt time.Time `json:"expires_at"`
+	}
+	if err := json.Unmarshal(body, &out); err != nil {
+		return fmt.Errorf("github app: decode token response: %w", err)
+	}
+	if out.Token == "" {
+		return errors.New("github app: token exchange returned an empty token")
+	}
+	a.token = out.Token
+	a.expiry = out.ExpiresAt
+	if a.expiry.IsZero() {
+		// Defensive: assume the documented ~1h lifetime if GitHub omits it.
+		a.expiry = time.Now().Add(time.Hour)
+	}
+	return nil
+}
+
+// mintJWT builds and RS256-signs a GitHub App JWT (iss=app id, backdated iat,
+// ≤10m exp) using stdlib crypto — no third-party JWT dependency.
+func mintJWT(appID string, key *rsa.PrivateKey, now time.Time) (string, error) {
+	header := map[string]string{"alg": "RS256", "typ": "JWT"}
+	claims := map[string]any{
+		"iat": now.Add(-jwtBackdate).Unix(),
+		"exp": now.Add(jwtLifetime).Unix(),
+		"iss": appID,
+	}
+	hb, err := json.Marshal(header)
+	if err != nil {
+		return "", err
+	}
+	cb, err := json.Marshal(claims)
+	if err != nil {
+		return "", err
+	}
+	signingInput := b64url(hb) + "." + b64url(cb)
+	digest := sha256.Sum256([]byte(signingInput))
+	sig, err := rsa.SignPKCS1v15(rand.Reader, key, crypto.SHA256, digest[:])
+	if err != nil {
+		return "", err
+	}
+	return signingInput + "." + b64url(sig), nil
+}
+
+func b64url(b []byte) string {
+	return base64.RawURLEncoding.EncodeToString(b)
+}
+
+// parseRSAPrivateKey accepts PKCS#1 ("RSA PRIVATE KEY") and PKCS#8 ("PRIVATE
+// KEY") PEM, covering both GitHub App key export formats.
+func parseRSAPrivateKey(pemBytes []byte) (*rsa.PrivateKey, error) {
+	block, _ := pem.Decode(pemBytes)
+	if block == nil {
+		return nil, errors.New("github app: private key is not valid PEM")
+	}
+	if key, err := x509.ParsePKCS1PrivateKey(block.Bytes); err == nil {
+		return key, nil
+	}
+	keyAny, err := x509.ParsePKCS8PrivateKey(block.Bytes)
+	if err != nil {
+		return nil, errors.New("github app: private key is not a supported RSA PKCS#1/PKCS#8 key")
+	}
+	rsaKey, ok := keyAny.(*rsa.PrivateKey)
+	if !ok {
+		return nil, errors.New("github app: private key is not an RSA key")
+	}
+	return rsaKey, nil
+}

internal/githubauth/app_test.go

@@ -0,0 +1,207 @@
+package githubauth
+
+import (
+	"context"
+	"crypto"
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/sha256"
+	"crypto/x509"
+	"encoding/base64"
+	"encoding/json"
+	"encoding/pem"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+)
+
+func testRSAKeyPEM(t *testing.T) string {
+	t.Helper()
+	key, err := rsa.GenerateKey(rand.Reader, 2048)
+	if err != nil {
+		t.Fatalf("generate key: %v", err)
+	}
+	der := x509.MarshalPKCS1PrivateKey(key)
+	return string(pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: der}))
+}
+
+// appFixture serves the installation-token exchange endpoint, records requests,
+// verifies the presented JWT against the app public key, and returns tokens with
+// a controllable expiry.
+type appFixture struct {
+	srv       *httptest.Server
+	pub       *rsa.PublicKey
+	mu        sync.Mutex
+	exchanges int
+	lastJWT   string
+	expiresAt func() time.Time
+	tokenSeq  int
+}
+
+func newAppFixture(t *testing.T, pemKey string) *appFixture {
+	t.Helper()
+	block, _ := pem.Decode([]byte(pemKey))
+	key, err := x509.ParsePKCS1PrivateKey(block.Bytes)
+	if err != nil {
+		t.Fatalf("parse test key: %v", err)
+	}
+	f := &appFixture{
+		pub:       &key.PublicKey,
+		expiresAt: func() time.Time { return time.Now().Add(time.Hour) },
+	}
+	mux := http.NewServeMux()
+	mux.HandleFunc("/app/installations/456/access_tokens", func(w http.ResponseWriter, r *http.Request) {
+		auth := r.Header.Get("Authorization")
+		jwt := strings.TrimPrefix(auth, "Bearer ")
+		f.mu.Lock()
+		f.exchanges++
+		f.lastJWT = jwt
+		f.tokenSeq++
+		seq := f.tokenSeq
+		exp := f.expiresAt()
+		f.mu.Unlock()
+		w.Header().Set("Content-Type", "application/json")
+		w.WriteHeader(http.StatusCreated)
+		json.NewEncoder(w).Encode(map[string]any{
+			"token":      fmt.Sprintf("ghs_installation_%d", seq),
+			"expires_at": exp.UTC().Format(time.RFC3339),
+		})
+	})
+	f.srv = httptest.NewServer(mux)
+	t.Cleanup(f.srv.Close)
+	return f
+}
+
+func (f *appFixture) verifyJWT(t *testing.T) {
+	t.Helper()
+	f.mu.Lock()
+	jwt := f.lastJWT
+	f.mu.Unlock()
+	parts := strings.Split(jwt, ".")
+	if len(parts) != 3 {
+		t.Fatalf("jwt not three-part: %q", jwt)
+	}
+	signingInput := parts[0] + "." + parts[1]
+	sig, err := base64.RawURLEncoding.DecodeString(parts[2])
+	if err != nil {
+		t.Fatalf("decode sig: %v", err)
+	}
+	digest := sha256.Sum256([]byte(signingInput))
+	if err := rsa.VerifyPKCS1v15(f.pub, crypto.SHA256, digest[:], sig); err != nil {
+		t.Fatalf("jwt signature invalid: %v", err)
+	}
+	var claims struct {
+		Iss string `json:"iss"`
+		Iat int64  `json:"iat"`
+		Exp int64  `json:"exp"`
+	}
+	cb, _ := base64.RawURLEncoding.DecodeString(parts[1])
+	if err := json.Unmarshal(cb, &claims); err != nil {
+		t.Fatalf("decode claims: %v", err)
+	}
+	if claims.Iss != "123" {
+		t.Fatalf("iss = %q, want 123", claims.Iss)
+	}
+	if claims.Exp-claims.Iat > int64((10*time.Minute)/time.Second) {
+		t.Fatalf("jwt lifetime exceeds 10m: iat=%d exp=%d", claims.Iat, claims.Exp)
+	}
+	if claims.Iat > time.Now().Unix() {
+		t.Fatalf("iat not backdated: %d", claims.Iat)
+	}
+}
+
+func newAppCred(t *testing.T, f *appFixture, pemKey string) *appCredential {
+	t.Helper()
+	c, err := newAppCredential(Options{
+		AppID:          "123",
+		InstallationID: "456",
+		PrivateKeyPEM:  pemKey,
+		apiBaseURL:     f.srv.URL,
+		httpClient:     f.srv.Client(),
+	})
+	if err != nil {
+		t.Fatalf("newAppCredential: %v", err)
+	}
+	return c
+}
+
+func TestApp_MintsJWTAndExchangesForInstallationToken(t *testing.T) {
+	pemKey := testRSAKeyPEM(t)
+	f := newAppFixture(t, pemKey)
+	c := newAppCred(t, f, pemKey)
+
+	tok, err := c.Token(context.Background())
+	if err != nil {
+		t.Fatalf("token: %v", err)
+	}
+	if tok != "ghs_installation_1" {
+		t.Fatalf("token = %q, want ghs_installation_1", tok)
+	}
+	if f.exchanges != 1 {
+		t.Fatalf("exchanges = %d, want 1", f.exchanges)
+	}
+	f.verifyJWT(t)
+}
+
+func TestApp_CachesInstallationToken(t *testing.T) {
+	pemKey := testRSAKeyPEM(t)
+	f := newAppFixture(t, pemKey)
+	c := newAppCred(t, f, pemKey)
+
+	for i := 0; i < 5; i++ {
+		if _, err := c.Token(context.Background()); err != nil {
+			t.Fatalf("token: %v", err)
+		}
+	}
+	if f.exchanges != 1 {
+		t.Fatalf("exchanges = %d, want 1 (token should be cached)", f.exchanges)
+	}
+}
+
+func TestApp_RefreshesNearExpiry(t *testing.T) {
+	pemKey := testRSAKeyPEM(t)
+	f := newAppFixture(t, pemKey)
+	// Token expires within refreshSkew, so every call must re-exchange.
+	f.expiresAt = func() time.Time { return time.Now().Add(2 * time.Minute) }
+	c := newAppCred(t, f, pemKey)
+
+	t1, err := c.Token(context.Background())
+	if err != nil {
+		t.Fatalf("token 1: %v", err)
+	}
+	t2, err := c.Token(context.Background())
+	if err != nil {
+		t.Fatalf("token 2: %v", err)
+	}
+	if f.exchanges != 2 {
+		t.Fatalf("exchanges = %d, want 2 (near-expiry token must refresh)", f.exchanges)
+	}
+	if t1 == t2 {
+		t.Fatalf("expected a fresh token after refresh, both = %q", t1)
+	}
+}
+
+func TestApp_ConcurrentTokenSingleFlights(t *testing.T) {
+	pemKey := testRSAKeyPEM(t)
+	f := newAppFixture(t, pemKey)
+	c := newAppCred(t, f, pemKey)
+
+	var wg sync.WaitGroup
+	for i := 0; i < 20; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			if _, err := c.Token(context.Background()); err != nil {
+				t.Errorf("token: %v", err)
+			}
+		}()
+	}
+	wg.Wait()
+	if f.exchanges != 1 {
+		t.Fatalf("exchanges = %d, want 1 (concurrent calls must coalesce)", f.exchanges)
+	}
+}

internal/githubauth/credential.go

@@ -0,0 +1,106 @@
+// Package githubauth provides the process-wide GitHub machine credential used to
+// authenticate every outbound GitHub request (releases scan, ranged asset header
+// fetches, and the generic-github byte proxy for private assets). The credential
+// is delivered via env/secret only — it is never stored per-remote in the DB,
+// never returned by any API, and never logged.
+package githubauth
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"os"
+	"strings"
+	"sync"
+)
+
+// Credential yields a bearer token for GitHub requests. Token may block to mint
+// or refresh (the GitHub App path); an empty string means "no auth", which only
+// happens when no credential is configured.
+type Credential interface {
+	Token(ctx context.Context) (string, error)
+}
+
+// Options is the raw, env-sourced auth configuration. Exactly one mode may be
+// configured: a static token, or a GitHub App (id + installation id + private
+// key). Partial App configuration is an error (fail closed); no fields at all is
+// fine and yields a nil credential (anonymous, current behavior).
+type Options struct {
+	// Token is a Personal Access Token (fine-grained or classic) sent verbatim
+	// as "Authorization: Bearer <token>".
+	Token string
+
+	// GitHub App fields. PrivateKeyPEM and PrivateKeyPath are alternatives; the
+	// inline PEM wins when both are set.
+	AppID          string
+	InstallationID string
+	PrivateKeyPEM  string
+	PrivateKeyPath string
+
+	// apiBaseURL overrides https://api.github.com for tests. Empty uses the real
+	// endpoint. httpClient likewise overrides the default client for tests.
+	apiBaseURL string
+	httpClient httpDoer
+}
+
+// New builds the process credential from options, validating that auth is either
+// fully configured or fully absent. It returns (nil, nil) when nothing is set.
+func New(opts Options) (Credential, error) {
+	hasToken := opts.Token != ""
+	hasAppField := opts.AppID != "" || opts.InstallationID != "" ||
+		opts.PrivateKeyPEM != "" || opts.PrivateKeyPath != ""
+
+	switch {
+	case !hasToken && !hasAppField:
+		return nil, nil // no auth configured — anonymous is fine
+	case hasToken && hasAppField:
+		return nil, errors.New("github auth: both a token and GitHub App fields are set; configure exactly one")
+	case hasToken:
+		return staticToken{token: opts.Token}, nil
+	default:
+		return newAppCredential(opts)
+	}
+}
+
+// staticToken is a fixed PAT credential.
+type staticToken struct{ token string }
+
+func (s staticToken) Token(context.Context) (string, error) { return s.token, nil }
+
+// server is the process-wide credential set once at startup. A nil value means
+// no server credential (anonymous). Access is guarded so a late SetServer in a
+// test is race-free.
+var (
+	serverMu sync.RWMutex
+	server   Credential
+)
+
+// SetServer installs the process credential. Call once during startup.
+func SetServer(c Credential) {
+	serverMu.Lock()
+	server = c
+	serverMu.Unlock()
+}
+
+// Server returns the process credential, or nil if none is configured.
+func Server() Credential {
+	serverMu.RLock()
+	defer serverMu.RUnlock()
+	return server
+}
+
+// loadPrivateKeyPEM resolves the App private key bytes from the inline PEM or a
+// file path, without ever returning the key material in an error message.
+func loadPrivateKeyPEM(opts Options) ([]byte, error) {
+	if strings.TrimSpace(opts.PrivateKeyPEM) != "" {
+		return []byte(opts.PrivateKeyPEM), nil
+	}
+	if opts.PrivateKeyPath != "" {
+		b, err := os.ReadFile(opts.PrivateKeyPath)
+		if err != nil {
+			return nil, fmt.Errorf("github app: read private key file: %w", err)
+		}
+		return b, nil
+	}
+	return nil, errors.New("github app: no private key configured")
+}

internal/githubauth/credential_test.go

@@ -0,0 +1,77 @@
+package githubauth
+
+import (
+	"context"
+	"testing"
+)
+
+func TestNew_NoConfigIsAnonymous(t *testing.T) {
+	c, err := New(Options{})
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if c != nil {
+		t.Fatalf("expected nil credential when nothing configured, got %T", c)
+	}
+}
+
+func TestNew_TokenMode(t *testing.T) {
+	c, err := New(Options{Token: "ghp_example"})
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	tok, err := c.Token(context.Background())
+	if err != nil {
+		t.Fatalf("token: %v", err)
+	}
+	if tok != "ghp_example" {
+		t.Fatalf("token = %q, want ghp_example", tok)
+	}
+}
+
+func TestNew_TokenAndAppConflict(t *testing.T) {
+	_, err := New(Options{Token: "ghp_example", AppID: "123"})
+	if err == nil {
+		t.Fatal("expected error when both token and app fields are set")
+	}
+}
+
+func TestNew_PartialAppFailsClosed(t *testing.T) {
+	cases := map[string]Options{
+		"app id without key":  {AppID: "123", InstallationID: "456"},
+		"key without app id":  {InstallationID: "456", PrivateKeyPEM: testRSAKeyPEM(t)},
+		"app id without inst": {AppID: "123", PrivateKeyPEM: testRSAKeyPEM(t)},
+	}
+	for name, opts := range cases {
+		t.Run(name, func(t *testing.T) {
+			if _, err := New(opts); err == nil {
+				t.Fatalf("expected fail-closed error for %q", name)
+			}
+		})
+	}
+}
+
+func TestNew_AppModeParsesKey(t *testing.T) {
+	c, err := New(Options{
+		AppID:          "123",
+		InstallationID: "456",
+		PrivateKeyPEM:  testRSAKeyPEM(t),
+	})
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if _, ok := c.(*appCredential); !ok {
+		t.Fatalf("expected *appCredential, got %T", c)
+	}
+}
+
+func TestNew_AppModeRejectsBadKey(t *testing.T) {
+	_, err := New(Options{
+		AppID:          "123",
+		InstallationID: "456",
+		PrivateKeyPEM:  "-----BEGIN RSA PRIVATE KEY-----\nnope\n-----END RSA PRIVATE KEY-----",
+	})
+	if err == nil {
+		t.Fatal("expected error for malformed private key")
+	}
+}

internal/provider/generic/generic.go

@@ -4,9 +4,11 @@ import (
 	"context"
 	"encoding/base64"
 	"net/http"
+	"net/url"
 	"path"
 	"strings"
 
+	"git.unkin.net/unkin/artifactapi/internal/githubauth"
 	"git.unkin.net/unkin/artifactapi/internal/provider"
 	"git.unkin.net/unkin/artifactapi/pkg/models"
 )
@@ -59,10 +61,42 @@ func (p *Provider) RewriteResponse(_ []byte, _ models.Remote, _ string) ([]byte,
 	return nil, nil
 }
 
-func (p *Provider) AuthHeaders(_ context.Context, remote models.Remote) (http.Header, error) {
+// AuthHeaders authenticates outbound requests. A per-remote username/password
+// (Basic auth) takes precedence. Otherwise, when the remote points at a GitHub
+// host (e.g. a releases_remote proxying private release assets), the process-wide
+// GitHub credential is attached as a bearer token so private downloads work.
+func (p *Provider) AuthHeaders(ctx context.Context, remote models.Remote) (http.Header, error) {
 	h := http.Header{}
 	if remote.Username != "" {
 		h.Set("Authorization", "Basic "+base64.StdEncoding.EncodeToString([]byte(remote.Username+":"+remote.Password)))
+		return h, nil
+	}
+	if isGitHubHost(remote.BaseURL) {
+		if c := githubauth.Server(); c != nil {
+			tok, err := c.Token(ctx)
+			if err != nil {
+				return nil, err
+			}
+			if tok != "" {
+				h.Set("Authorization", "Bearer "+tok)
+			}
+		}
 	}
 	return h, nil
 }
+
+// isGitHubHost reports whether rawURL targets a GitHub API/download host that
+// accepts the server credential. objects.githubusercontent.com is deliberately
+// excluded: release-asset downloads 302-redirect there with a pre-signed URL
+// that must not carry an Authorization header.
+func isGitHubHost(rawURL string) bool {
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		return false
+	}
+	switch strings.ToLower(u.Hostname()) {
+	case "github.com", "www.github.com", "api.github.com", "codeload.github.com", "uploads.github.com":
+		return true
+	}
+	return false
+}

internal/provider/generic/generic_test.go

@@ -4,11 +4,56 @@ import (
 	"context"
 	"testing"
 
+	"git.unkin.net/unkin/artifactapi/internal/githubauth"
 	"git.unkin.net/unkin/artifactapi/internal/provider"
 	"git.unkin.net/unkin/artifactapi/internal/provider/generic"
 	"git.unkin.net/unkin/artifactapi/pkg/models"
 )
 
+type staticCred string
+
+func (s staticCred) Token(context.Context) (string, error) { return string(s), nil }
+
+func TestProvider_AuthHeaders_GitHubServerCredential(t *testing.T) {
+	githubauth.SetServer(staticCred("ghs_server"))
+	t.Cleanup(func() { githubauth.SetServer(nil) })
+
+	p := &generic.Provider{}
+	h, err := p.AuthHeaders(context.Background(), models.Remote{BaseURL: "https://github.com"})
+	if err != nil {
+		t.Fatalf("auth headers: %v", err)
+	}
+	if h.Get("Authorization") != "Bearer ghs_server" {
+		t.Fatalf("Authorization = %q, want Bearer ghs_server", h.Get("Authorization"))
+	}
+}
+
+func TestProvider_AuthHeaders_NonGitHubHostNoServerCredential(t *testing.T) {
+	githubauth.SetServer(staticCred("ghs_server"))
+	t.Cleanup(func() { githubauth.SetServer(nil) })
+
+	p := &generic.Provider{}
+	h, _ := p.AuthHeaders(context.Background(), models.Remote{BaseURL: "https://example.com/downloads"})
+	if h.Get("Authorization") != "" {
+		t.Fatalf("server credential must not be sent to non-github host, got %q", h.Get("Authorization"))
+	}
+}
+
+func TestProvider_AuthHeaders_PerRemoteOverridesServerCredential(t *testing.T) {
+	githubauth.SetServer(staticCred("ghs_server"))
+	t.Cleanup(func() { githubauth.SetServer(nil) })
+
+	p := &generic.Provider{}
+	h, _ := p.AuthHeaders(context.Background(), models.Remote{
+		BaseURL:  "https://github.com",
+		Username: "user",
+		Password: "pass",
+	})
+	if got := h.Get("Authorization"); got != "Basic dXNlcjpwYXNz" {
+		t.Fatalf("per-remote Basic auth must win, got %q", got)
+	}
+}
+
 func TestProvider_Type(t *testing.T) {
 	p := &generic.Provider{}
 	if p.Type() != models.PackageGeneric {

internal/provider/rpm/github.go

@@ -18,13 +18,20 @@ import (
 	"time"
 
 	rpmlib "github.com/cavaliergopher/rpm"
+	"golang.org/x/time/rate"
 
+	"git.unkin.net/unkin/artifactapi/internal/githubauth"
 	"git.unkin.net/unkin/artifactapi/internal/provider"
 	"git.unkin.net/unkin/artifactapi/pkg/models"
 )
 
+// gitHubProvider is the process-wide singleton. The background Syncer binds its
+// shared rate limiter and work queue onto this instance so the request path and
+// the syncer drive the same derive machinery.
+var gitHubProvider = newGitHubProvider()
+
 func init() {
-	provider.Register(newGitHubProvider())
+	provider.Register(gitHubProvider)
 }
 
 // Tuning knobs for the no-precache header fetch. Fields (not consts) so tests
@@ -39,6 +46,12 @@ const (
 	defaultScanTimeout = 10 * time.Minute
 	// defaultServeTimeout bounds a repodata DB read served on a detached context.
 	defaultServeTimeout = 30 * time.Second
+
+	// defaultColdWait bounds how long a repodata request blocks waiting for a
+	// just-enqueued prime to populate an empty cache before returning a
+	// retryable 503. Kept short so a client never hangs on a rate-limited derive
+	// of a large repo; small repos usually prime within this window.
+	defaultColdWait = 8 * time.Second
 )
 
 // GitHubProvider is a metadata-only remote: it scans a GitHub repo's releases
@@ -54,6 +67,19 @@ type GitHubProvider struct {
 	pageCap       int
 	scanTimeout   time.Duration
 	serveTimeout  time.Duration
+	coldWait      time.Duration
+
+	// limiter, when set by the Syncer, gates every GitHub HTTP call (releases
+	// list + each ranged asset fetch) through a single process-wide token bucket.
+	// nil means unlimited (direct provider use / unit tests).
+	limiter *rate.Limiter
+	// syncer, when set, routes freshness refresh and cold-start priming through
+	// the shared background work queue instead of an inline per-replica scan.
+	syncer *Syncer
+
+	// serverCred overrides the process-wide GitHub credential for this provider
+	// instance. nil falls back to githubauth.Server(); set directly in tests.
+	serverCred githubauth.Credential
 
 	mu       sync.Mutex
 	scanning map[string]bool
@@ -68,11 +94,21 @@ func newGitHubProvider() *GitHubProvider {
 		pageCap:       defaultReleasePageCap,
 		scanTimeout:   defaultScanTimeout,
 		serveTimeout:  defaultServeTimeout,
+		coldWait:      defaultColdWait,
 		scanning:      map[string]bool{},
 		lastScan:      map[string]time.Time{},
 	}
 }
 
+// limiterWait blocks until the shared rate limiter grants a token, or returns
+// the context error if it is canceled first. A nil limiter is a no-op.
+func (p *GitHubProvider) limiterWait(ctx context.Context) error {
+	if p.limiter == nil {
+		return nil
+	}
+	return p.limiter.Wait(ctx)
+}
+
 func (p *GitHubProvider) Type() models.PackageType { return models.PackageGitHubRPM }
 
 // Classify/ContentType/UpstreamURL/RewriteResponse/AuthHeaders satisfy the
@@ -105,8 +141,8 @@ func (p *GitHubProvider) RewriteResponse(_ []byte, _ models.Remote, _ string) ([
 	return nil, nil
 }
 
-func (p *GitHubProvider) AuthHeaders(_ context.Context, remote models.Remote) (http.Header, error) {
-	return githubHeaders(remote, false), nil
+func (p *GitHubProvider) AuthHeaders(ctx context.Context, remote models.Remote) (http.Header, error) {
+	return p.githubHeaders(ctx, remote, false)
 }
 
 // ServeRemote answers a request against a github_rpm remote. It refreshes the
@@ -114,7 +150,7 @@ func (p *GitHubProvider) AuthHeaders(_ context.Context, remote models.Remote) (h
 // 302-redirects .rpm downloads to the backend releases_remote. Returns false
 // only for paths it does not own, letting the normal proxy path take over.
 func (p *GitHubProvider) ServeRemote(w http.ResponseWriter, r *http.Request, remote models.Remote, path, proxyBaseURL string, store provider.RemoteMetadataStore) bool {
-	p.refresh(remote, store)
+	p.onRequest(remote, store)
 
 	if strings.HasPrefix(path, "repodata/") {
 		// Serve repodata on a context detached from the inbound request: a
@@ -124,6 +160,15 @@ func (p *GitHubProvider) ServeRemote(w http.ResponseWriter, r *http.Request, rem
 		defer cancel()
 		sr := r.WithContext(sctx)
 
+		// Cold start: with the syncer wired, an empty cache means the prime has
+		// not landed yet. Enqueue it and wait briefly rather than serving empty
+		// repodata; if it still has not primed, return a retryable 503.
+		if p.syncer != nil && !p.ensurePrimed(sctx, remote, store) {
+			w.Header().Set("Retry-After", "5")
+			http.Error(w, "metadata is being prepared, retry shortly", http.StatusServiceUnavailable)
+			return true
+		}
+
 		tail := strings.TrimPrefix(path, "repodata/")
 		lp := &Provider{}
 		switch {
@@ -154,6 +199,56 @@ func (p *GitHubProvider) ServeRemote(w http.ResponseWriter, r *http.Request, rem
 	return false
 }
 
+// onRequest keeps a remote's derived metadata fresh off the request path. With
+// the background syncer wired it enqueues a deduped, rate-limited, lease-gated
+// refresh and returns immediately; the request always serves the current cache.
+// Without a syncer (direct provider use / unit tests) it falls back to the
+// legacy inline single-flight scan.
+func (p *GitHubProvider) onRequest(remote models.Remote, store provider.RemoteMetadataStore) {
+	if p.syncer != nil {
+		p.syncer.enqueue(remote, false)
+		return
+	}
+	p.refresh(remote, store)
+}
+
+// ensurePrimed returns true once the remote has at least one cached metadata
+// row. On an empty cache it enqueues a prime and polls briefly for it to land,
+// so the very first client after a remote is created gets real repodata instead
+// of an empty index or a blocking multi-minute derive. Returns false if the
+// cache is still empty after the bounded wait.
+func (p *GitHubProvider) ensurePrimed(ctx context.Context, remote models.Remote, store provider.RemoteMetadataStore) bool {
+	if !p.cacheEmpty(ctx, store, remote.Name) {
+		return true
+	}
+	if p.syncer != nil {
+		p.syncer.enqueue(remote, true)
+	}
+
+	deadline := time.Now().Add(p.coldWait)
+	for time.Now().Before(deadline) {
+		select {
+		case <-ctx.Done():
+			return false
+		case <-time.After(400 * time.Millisecond):
+		}
+		if !p.cacheEmpty(ctx, store, remote.Name) {
+			return true
+		}
+	}
+	return false
+}
+
+func (p *GitHubProvider) cacheEmpty(ctx context.Context, store provider.RemoteMetadataStore, name string) bool {
+	rows, err := store.ListRPMMetadataEntries(ctx, name)
+	if err != nil {
+		// Treat a failed read as "not empty" so a transient DB error becomes a
+		// normal serve attempt (which reports its own error) rather than a 503.
+		return false
+	}
+	return len(rows) == 0
+}
+
 // refresh brings the derived metadata up to date without coupling the scan to
 // the inbound request. When the cache is stale it single-flights a scan: if the
 // cache already holds rows the scan runs in the background and the caller serves
@@ -212,15 +307,31 @@ func (p *GitHubProvider) runScan(remote models.Remote, store provider.RemoteMeta
 	p.mu.Unlock()
 }
 
+// scan runs a full unconditional derive. Retained for the legacy inline refresh
+// path and existing tests; the syncer uses scanWithState to pass and receive the
+// releases-list ETag.
 func (p *GitHubProvider) scan(ctx context.Context, remote models.Remote, store provider.RemoteMetadataStore) error {
-	releases, err := p.fetchReleases(ctx, remote)
+	_, _, err := p.scanWithState(ctx, remote, store, "")
+	return err
+}
+
+// scanWithState derives metadata incrementally. It sends the prior releases-list
+// ETag as a conditional request: a 304 means nothing changed, so it returns
+// (etag, changed=false) without a single asset fetch. On a 200 it diffs the
+// release assets against the cache, derives only new/changed assets, prunes
+// assets that disappeared, and returns the new ETag.
+func (p *GitHubProvider) scanWithState(ctx context.Context, remote models.Remote, store provider.RemoteMetadataStore, etag string) (newEtag string, changed bool, err error) {
+	releases, newEtag, notModified, err := p.fetchReleases(ctx, remote, etag)
 	if err != nil {
-		return err
+		return etag, false, err
+	}
+	if notModified {
+		return etag, false, nil
 	}
 
 	existing, err := store.ListRPMMetadataEntries(ctx, remote.Name)
 	if err != nil {
-		return err
+		return newEtag, false, err
 	}
 	existingByPath := make(map[string]provider.RPMMetadata, len(existing))
 	for _, m := range existing {
@@ -229,7 +340,7 @@ func (p *GitHubProvider) scan(ctx context.Context, remote models.Remote, store p
 
 	allow, err := compilePatterns(remote.Patterns)
 	if err != nil {
-		return err
+		return newEtag, false, err
 	}
 
 	seen := map[string]bool{}
@@ -277,7 +388,7 @@ func (p *GitHubProvider) scan(ctx context.Context, remote models.Remote, store p
 			_ = store.DeleteRPMMetadata(ctx, remote.Name, fp)
 		}
 	}
-	return nil
+	return newEtag, true, nil
 }
 
 type ghRelease struct {
@@ -293,32 +404,57 @@ type ghAsset struct {
 	Digest             string `json:"digest"`
 }
 
-func (p *GitHubProvider) fetchReleases(ctx context.Context, remote models.Remote) ([]ghRelease, error) {
+// fetchReleases lists a repo's releases. It sends the prior ETag as
+// If-None-Match on page 1 (the newest releases, where a new one first appears):
+// a 304 there means the repo is unchanged, so it returns notModified without
+// paging further — GitHub does not count 304 conditional responses against the
+// rate limit, making an unchanged repo nearly free. On a 200 it captures the
+// page-1 ETag and pages through the rest normally. Every call waits on the
+// shared limiter first.
+func (p *GitHubProvider) fetchReleases(ctx context.Context, remote models.Remote, etag string) (all []ghRelease, newEtag string, notModified bool, err error) {
 	base := strings.TrimRight(remote.BaseURL, "/") + "/releases"
-	var all []ghRelease
 	for page := 1; page <= p.pageCap; page++ {
 		u := fmt.Sprintf("%s?per_page=100&page=%d", base, page)
 		req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
 		if err != nil {
-			return nil, err
+			return nil, "", false, err
+		}
+		hdr, err := p.githubHeaders(ctx, remote, true)
+		if err != nil {
+			return nil, "", false, err
+		}
+		copyHeaders(req, hdr)
+		if page == 1 && etag != "" {
+			req.Header.Set("If-None-Match", etag)
 		}
-		copyHeaders(req, githubHeaders(remote, true))
 
+		if err := p.limiterWait(ctx); err != nil {
+			return nil, "", false, err
+		}
 		resp, err := p.client.Do(req)
 		if err != nil {
-			return nil, err
+			return nil, "", false, err
+		}
+		if page == 1 && resp.StatusCode == http.StatusNotModified {
+			io.Copy(io.Discard, resp.Body)
+			resp.Body.Close()
+			return nil, etag, true, nil
 		}
 		body, err := io.ReadAll(resp.Body)
+		respEtag := resp.Header.Get("ETag")
 		resp.Body.Close()
 		if err != nil {
-			return nil, err
+			return nil, "", false, err
 		}
 		if resp.StatusCode != http.StatusOK {
-			return nil, fmt.Errorf("github releases API %s: status %d", u, resp.StatusCode)
+			return nil, "", false, fmt.Errorf("github releases API %s: status %d", u, resp.StatusCode)
+		}
+		if page == 1 {
+			newEtag = respEtag
 		}
 		var releases []ghRelease
 		if err := json.Unmarshal(body, &releases); err != nil {
-			return nil, fmt.Errorf("decode releases: %w", err)
+			return nil, "", false, fmt.Errorf("decode releases: %w", err)
 		}
 		if len(releases) == 0 {
 			break
@@ -328,7 +464,7 @@ func (p *GitHubProvider) fetchReleases(ctx context.Context, remote models.Remote
 			break
 		}
 	}
-	return all, nil
+	return all, newEtag, false, nil
 }
 
 func (p *GitHubProvider) deriveAsset(ctx context.Context, remote models.Remote, asset ghAsset, fp string) (*provider.RPMMetadata, error) {
@@ -444,9 +580,16 @@ func (p *GitHubProvider) rangeGet(ctx context.Context, remote models.Remote, dow
 	if err != nil {
 		return nil, false, err
 	}
-	copyHeaders(req, githubHeaders(remote, false))
+	hdr, err := p.githubHeaders(ctx, remote, false)
+	if err != nil {
+		return nil, false, err
+	}
+	copyHeaders(req, hdr)
 	req.Header.Set("Range", fmt.Sprintf("bytes=0-%d", n-1))
 
+	if err := p.limiterWait(ctx); err != nil {
+		return nil, false, err
+	}
 	resp, err := p.client.Do(req)
 	if err != nil {
 		return nil, false, err
@@ -469,8 +612,15 @@ func (p *GitHubProvider) computeSHA256(ctx context.Context, remote models.Remote
 	if err != nil {
 		return "", err
 	}
-	copyHeaders(req, githubHeaders(remote, false))
+	hdr, err := p.githubHeaders(ctx, remote, false)
+	if err != nil {
+		return "", err
+	}
+	copyHeaders(req, hdr)
 
+	if err := p.limiterWait(ctx); err != nil {
+		return "", err
+	}
 	resp, err := p.client.Do(req)
 	if err != nil {
 		return "", err
@@ -505,23 +655,48 @@ func sha256FromDigest(digest string) (string, bool) {
 	return "", false
 }
 
-func githubHeaders(remote models.Remote, api bool) http.Header {
+// githubHeaders builds the outbound headers for a GitHub request, attaching a
+// bearer credential when one is available. A per-remote credential wins; absent
+// that, the process-wide server credential is used; absent both, the request is
+// unauthenticated (anonymous, subject to the 60/hr cap).
+func (p *GitHubProvider) githubHeaders(ctx context.Context, remote models.Remote, api bool) (http.Header, error) {
 	h := http.Header{}
 	if api {
 		h.Set("Accept", "application/vnd.github+json")
 		h.Set("X-GitHub-Api-Version", "2022-11-28")
 	}
-	if tok := githubToken(remote); tok != "" {
+	tok, err := p.githubToken(ctx, remote)
+	if err != nil {
+		return nil, err
+	}
+	if tok != "" {
 		h.Set("Authorization", "Bearer "+tok)
 	}
-	return h
+	return h, nil
 }
 
-func githubToken(remote models.Remote) string {
+// githubToken resolves the bearer token for a remote. Precedence: a per-remote
+// credential (password, then username) overrides the server credential.
+func (p *GitHubProvider) githubToken(ctx context.Context, remote models.Remote) (string, error) {
 	if remote.Password != "" {
-		return remote.Password
+		return remote.Password, nil
 	}
-	return remote.Username
+	if remote.Username != "" {
+		return remote.Username, nil
+	}
+	if c := p.serverCredential(); c != nil {
+		return c.Token(ctx)
+	}
+	return "", nil
+}
+
+// serverCredential returns this provider's server credential, defaulting to the
+// process-wide one installed at startup.
+func (p *GitHubProvider) serverCredential() githubauth.Credential {
+	if p.serverCred != nil {
+		return p.serverCred
+	}
+	return githubauth.Server()
 }
 
 func copyHeaders(req *http.Request, h http.Header) {

internal/provider/rpm/github_auth_test.go

@@ -0,0 +1,134 @@
+package rpm
+
+import (
+	"context"
+	"encoding/json"
+	"strings"
+	"testing"
+
+	"git.unkin.net/unkin/artifactapi/internal/githubauth"
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+// staticCred is a test Credential yielding a fixed token.
+type staticCred string
+
+func (s staticCred) Token(context.Context) (string, error) { return string(s), nil }
+
+func TestGitHubServerCredentialAttachedToReleasesAndAssets(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	p := newTestProvider()
+	p.serverCred = staticCred("ghp_server_secret")
+	store := newFakeStore()
+
+	if err := p.scan(context.Background(), fx.remote(), store); err != nil {
+		t.Fatalf("scan: %v", err)
+	}
+	if got := fx.releaseAuth; got != "Bearer ghp_server_secret" {
+		t.Fatalf("releases Authorization = %q, want Bearer ghp_server_secret", got)
+	}
+	if got := fx.assetAuth; got != "Bearer ghp_server_secret" {
+		t.Fatalf("asset Authorization = %q, want Bearer ghp_server_secret", got)
+	}
+}
+
+func TestGitHubPerRemoteCredentialOverridesServer(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	p := newTestProvider()
+	p.serverCred = staticCred("ghp_server_secret")
+	store := newFakeStore()
+
+	remote := fx.remote()
+	remote.Password = "ghp_remote_wins"
+
+	if err := p.scan(context.Background(), remote, store); err != nil {
+		t.Fatalf("scan: %v", err)
+	}
+	if got := fx.releaseAuth; got != "Bearer ghp_remote_wins" {
+		t.Fatalf("releases Authorization = %q, want per-remote token to win", got)
+	}
+	if got := fx.assetAuth; got != "Bearer ghp_remote_wins" {
+		t.Fatalf("asset Authorization = %q, want per-remote token to win", got)
+	}
+}
+
+func TestGitHubNoCredentialSendsNoAuthHeader(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	p := newTestProvider() // serverCred nil, package Server() unset in unit tests
+	store := newFakeStore()
+
+	if err := p.scan(context.Background(), fx.remote(), store); err != nil {
+		t.Fatalf("scan: %v", err)
+	}
+	if fx.releaseAuth != "" {
+		t.Fatalf("expected no Authorization header, got %q", fx.releaseAuth)
+	}
+	if fx.assetAuth != "" {
+		t.Fatalf("expected no asset Authorization header, got %q", fx.assetAuth)
+	}
+	// Requests still succeed anonymously.
+	if rows, _ := store.ListRPMMetadataEntries(context.Background(), "acme-rpm"); len(rows) != 1 {
+		t.Fatalf("anonymous scan should still derive metadata, got %d rows", len(rows))
+	}
+}
+
+func TestGitHubETag304FlowWithAuth(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	fx.etag = `"v1"`
+	p := newTestProvider()
+	p.serverCred = staticCred("ghp_server_secret")
+	store := newFakeStore()
+
+	etag, changed, err := p.scanWithState(context.Background(), fx.remote(), store, "")
+	if err != nil {
+		t.Fatalf("first scan: %v", err)
+	}
+	if !changed || etag != `"v1"` {
+		t.Fatalf("first scan changed=%v etag=%q, want true and \"v1\"", changed, etag)
+	}
+
+	// Re-scan with the captured ETag: a 304 means no change and no asset fetch.
+	etag2, changed2, err := p.scanWithState(context.Background(), fx.remote(), store, etag)
+	if err != nil {
+		t.Fatalf("second scan: %v", err)
+	}
+	if changed2 {
+		t.Fatal("expected no change on 304")
+	}
+	if etag2 != `"v1"` {
+		t.Fatalf("etag = %q, want preserved \"v1\"", etag2)
+	}
+	if fx.notModHit != 1 {
+		t.Fatalf("expected exactly one 304 response, got %d", fx.notModHit)
+	}
+	// The conditional request still carried the credential.
+	if fx.releaseAuth != "Bearer ghp_server_secret" {
+		t.Fatalf("conditional request Authorization = %q, want the server credential", fx.releaseAuth)
+	}
+}
+
+// TestGitHubCredentialAbsentFromRemoteJSON asserts the server credential never
+// appears in a remote's serialized API representation, and per-remote secrets
+// stay redacted by the models.Remote json:"-" tags.
+func TestGitHubCredentialAbsentFromRemoteJSON(t *testing.T) {
+	githubauth.SetServer(staticCred("ghp_super_secret_server_token"))
+	t.Cleanup(func() { githubauth.SetServer(nil) })
+
+	remote := models.Remote{
+		Name:        "acme-rpm",
+		PackageType: models.PackageGitHubRPM,
+		BaseURL:     "https://api.github.com/repos/acme/tools",
+		Username:    "per_remote_user",
+		Password:    "per_remote_secret",
+	}
+	b, err := json.Marshal(remote)
+	if err != nil {
+		t.Fatalf("marshal remote: %v", err)
+	}
+	js := string(b)
+	for _, secret := range []string{"ghp_super_secret_server_token", "per_remote_secret", "per_remote_user"} {
+		if strings.Contains(js, secret) {
+			t.Fatalf("credential %q leaked into remote JSON: %s", secret, js)
+		}
+	}
+}

internal/provider/rpm/github_test.go

@@ -66,11 +66,16 @@ func (f *fakeStore) ListRPMMetadataEntries(ctx context.Context, _ string) ([]pro
 // Range support) for a set of packages. digest controls whether the asset
 // carries a sha256 digest (no-download path) or not (compute path).
 type githubFixture struct {
-	srv      *httptest.Server
-	rpmBytes map[string][]byte // asset filename -> bytes
-	rangeHit map[string]int    // asset filename -> number of ranged GETs
-	fullHit  map[string]int    // asset filename -> number of full GETs
-	mu       sync.Mutex
+	srv         *httptest.Server
+	rpmBytes    map[string][]byte // asset filename -> bytes
+	rangeHit    map[string]int    // asset filename -> number of ranged GETs
+	fullHit     map[string]int    // asset filename -> number of full GETs
+	etag        string            // when set, served as ETag; matching If-None-Match yields 304
+	releasesHit int               // total releases-list requests (200 + 304)
+	notModHit   int               // releases-list requests answered 304
+	releaseAuth string            // Authorization header seen on the last releases request
+	assetAuth   string            // Authorization header seen on the last asset request
+	mu          sync.Mutex
 }
 
 func newGitHubFixture(t *testing.T, withDigest bool) *githubFixture {
@@ -89,6 +94,20 @@ func newGitHubFixture(t *testing.T, withDigest bool) *githubFixture {
 			w.Write([]byte("[]"))
 			return
 		}
+		f.mu.Lock()
+		f.releasesHit++
+		f.releaseAuth = r.Header.Get("Authorization")
+		etag := f.etag
+		if etag != "" && r.Header.Get("If-None-Match") == etag {
+			f.notModHit++
+			f.mu.Unlock()
+			w.WriteHeader(http.StatusNotModified)
+			return
+		}
+		f.mu.Unlock()
+		if etag != "" {
+			w.Header().Set("ETag", etag)
+		}
 		var assets []map[string]any
 		for name := range f.rpmBytes {
 			a := map[string]any{
@@ -114,6 +133,7 @@ func newGitHubFixture(t *testing.T, withDigest bool) *githubFixture {
 		}
 		rng := r.Header.Get("Range")
 		f.mu.Lock()
+		f.assetAuth = r.Header.Get("Authorization")
 		if rng != "" {
 			f.rangeHit[name]++
 		} else {

internal/provider/rpm/syncer.go

@@ -0,0 +1,256 @@
+package rpm
+
+import (
+	"context"
+	"crypto/rand"
+	"encoding/hex"
+	"log/slog"
+	"os"
+	"sync"
+	"time"
+
+	"golang.org/x/time/rate"
+
+	"git.unkin.net/unkin/artifactapi/internal/provider"
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+const (
+	// syncLeaseDuration is how long a claimed sync lease is held before it is
+	// considered abandoned. It comfortably exceeds a scan's own timeout so a live
+	// scan never loses its lease, while a crashed replica's lease still expires.
+	syncLeaseDuration = 15 * time.Minute
+	// defaultSyncFreshness is the periodic re-check interval used when a remote's
+	// mutable_ttl is unset.
+	defaultSyncFreshness = 5 * time.Minute
+	// jobQueueDepth bounds the pending work queue; enqueues past it are dropped
+	// (a later poll re-enqueues), never blocking the caller.
+	jobQueueDepth = 256
+)
+
+// SyncStore is the persistence surface the syncer needs: the metadata cache it
+// primes plus the shared sync-state coordination (remote enumeration and the
+// per-remote lease). *database.DB satisfies it.
+type SyncStore interface {
+	provider.RemoteMetadataStore
+	ListGitHubRPMRemotes(ctx context.Context) ([]models.Remote, error)
+	ClaimGitHubSyncLease(ctx context.Context, remoteName, owner string, freshness, lease time.Duration) (claimed bool, etag string, err error)
+	ReleaseGitHubSyncLease(ctx context.Context, remoteName, owner, etag string, syncedAt time.Time) error
+}
+
+// SyncConfig tunes the shared syncer. Zero values fall back to safe defaults.
+type SyncConfig struct {
+	RatePerSec   float64       // global GitHub request rate (req/s)
+	Burst        int           // token-bucket burst
+	Workers      int           // concurrent scan workers
+	PollInterval time.Duration // base scheduler tick; per-remote cadence is mutable_ttl
+}
+
+type syncJob struct {
+	remote models.Remote
+	prime  bool
+}
+
+// Syncer is the single per-process background worker that keeps every
+// github_rpm remote's derived metadata fresh. It owns a deduped work queue, a
+// pool of workers, and a global token-bucket rate limiter shared across all
+// remotes and bound onto the github provider so every GitHub call it makes
+// passes through the same bucket. Periodic checks are gated by a shared DB lease
+// so, across replicas, only one performs each scan.
+type Syncer struct {
+	store   SyncStore
+	prov    *GitHubProvider
+	limiter *rate.Limiter
+	cfg     SyncConfig
+	owner   string
+
+	jobs   chan syncJob
+	mu     sync.Mutex
+	active map[string]bool // remotes queued or in-flight, for dedup/coalescing
+}
+
+// NewSyncer builds the syncer bound to the process-wide github provider
+// singleton. Call Run to start it.
+func NewSyncer(store SyncStore, cfg SyncConfig) *Syncer {
+	return newSyncer(store, gitHubProvider, cfg)
+}
+
+func newSyncer(store SyncStore, prov *GitHubProvider, cfg SyncConfig) *Syncer {
+	if cfg.RatePerSec <= 0 {
+		cfg.RatePerSec = 1
+	}
+	if cfg.Burst <= 0 {
+		cfg.Burst = 5
+	}
+	if cfg.Workers <= 0 {
+		cfg.Workers = 3
+	}
+	if cfg.PollInterval <= 0 {
+		cfg.PollInterval = 60 * time.Second
+	}
+
+	lim := rate.NewLimiter(rate.Limit(cfg.RatePerSec), cfg.Burst)
+	s := &Syncer{
+		store:   store,
+		prov:    prov,
+		limiter: lim,
+		cfg:     cfg,
+		owner:   leaseOwner(),
+		jobs:    make(chan syncJob, jobQueueDepth),
+		active:  map[string]bool{},
+	}
+	// Bind the shared limiter and back-reference so the request path routes
+	// through this syncer and every derive HTTP call is rate limited.
+	prov.limiter = lim
+	prov.syncer = s
+	return s
+}
+
+// Run starts the worker pool and the periodic scheduler and blocks until ctx is
+// canceled, at which point it drains in-flight scans and returns.
+func (s *Syncer) Run(ctx context.Context) {
+	slog.Info("github_rpm syncer started",
+		"rate_per_sec", s.cfg.RatePerSec, "burst", s.cfg.Burst,
+		"workers", s.cfg.Workers, "poll_interval", s.cfg.PollInterval, "owner", s.owner)
+
+	var wg sync.WaitGroup
+	for i := 0; i < s.cfg.Workers; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			s.worker(ctx)
+		}()
+	}
+
+	ticker := time.NewTicker(s.cfg.PollInterval)
+	defer ticker.Stop()
+
+	s.schedule(ctx) // sweep at boot so existing remotes are checked immediately
+	for {
+		select {
+		case <-ctx.Done():
+			wg.Wait()
+			slog.Info("github_rpm syncer stopped")
+			return
+		case <-ticker.C:
+			s.schedule(ctx)
+		}
+	}
+}
+
+// schedule enqueues a periodic check for every github_rpm remote. The DB lease
+// (claimed in the worker) enforces the per-remote mutable_ttl cadence and cross
+// replica coordination, so enqueuing every tick is cheap: a not-yet-due remote
+// simply fails to claim and is skipped.
+func (s *Syncer) schedule(ctx context.Context) {
+	remotes, err := s.store.ListGitHubRPMRemotes(ctx)
+	if err != nil {
+		slog.Error("github_rpm syncer: list remotes", "error", err)
+		return
+	}
+	for _, r := range remotes {
+		s.enqueue(r, false)
+	}
+}
+
+// EnqueuePrime queues an immediate background prime for a freshly created
+// remote so its metadata is derived without blocking the create call.
+func (s *Syncer) EnqueuePrime(remote models.Remote) {
+	if s == nil {
+		return
+	}
+	s.enqueue(remote, true)
+}
+
+// enqueue adds a job unless the remote is already queued or in-flight, coalescing
+// duplicate requests down to one scan. It never blocks: a full queue drops the
+// job (a later poll re-enqueues it) after clearing the dedup slot.
+func (s *Syncer) enqueue(remote models.Remote, prime bool) {
+	s.mu.Lock()
+	if s.active[remote.Name] {
+		s.mu.Unlock()
+		return
+	}
+	s.active[remote.Name] = true
+	s.mu.Unlock()
+
+	select {
+	case s.jobs <- syncJob{remote: remote, prime: prime}:
+	default:
+		s.mu.Lock()
+		delete(s.active, remote.Name)
+		s.mu.Unlock()
+	}
+}
+
+func (s *Syncer) worker(ctx context.Context) {
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case job := <-s.jobs:
+			s.process(ctx, job)
+		}
+	}
+}
+
+// process claims the shared lease and, if won, runs an incremental scan. The
+// lease bounds total GitHub load to one scan per freshness window across all
+// replicas; losing the claim (another replica scanning, or not yet due) is a
+// no-op.
+func (s *Syncer) process(ctx context.Context, job syncJob) {
+	defer func() {
+		s.mu.Lock()
+		delete(s.active, job.remote.Name)
+		s.mu.Unlock()
+	}()
+
+	freshness := time.Duration(job.remote.MutableTTL) * time.Second
+	if freshness <= 0 {
+		freshness = defaultSyncFreshness
+	}
+	if job.prime {
+		freshness = 0 // prime ignores the recency gate but still respects a live lease
+	}
+
+	claimed, etag, err := s.store.ClaimGitHubSyncLease(ctx, job.remote.Name, s.owner, freshness, syncLeaseDuration)
+	if err != nil {
+		slog.Error("github_rpm syncer: claim lease", "remote", job.remote.Name, "error", err)
+		return
+	}
+	if !claimed {
+		return
+	}
+
+	scanCtx, cancel := context.WithTimeout(ctx, s.prov.scanTimeout)
+	defer cancel()
+
+	newEtag, changed, scanErr := s.prov.scanWithState(scanCtx, job.remote, s.store, etag)
+	releaseEtag := etag
+	if scanErr == nil {
+		releaseEtag = newEtag
+	} else {
+		slog.Error("github_rpm syncer: scan failed", "remote", job.remote.Name, "error", scanErr)
+	}
+
+	// Release on a detached context so a clean shutdown mid-scan still frees the
+	// lease and advances last_synced_at (otherwise it simply expires).
+	relCtx, relCancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
+	defer relCancel()
+	if err := s.store.ReleaseGitHubSyncLease(relCtx, job.remote.Name, s.owner, releaseEtag, time.Now()); err != nil {
+		slog.Warn("github_rpm syncer: release lease", "remote", job.remote.Name, "error", err)
+	}
+
+	if scanErr == nil && changed {
+		slog.Info("github_rpm syncer: refreshed", "remote", job.remote.Name, "prime", job.prime)
+	}
+}
+
+// leaseOwner is a per-replica identity for the lease: hostname plus a random
+// suffix so restarts and colocated replicas never collide.
+func leaseOwner() string {
+	host, _ := os.Hostname()
+	var b [6]byte
+	_, _ = rand.Read(b[:])
+	return host + "-" + hex.EncodeToString(b[:])
+}

internal/provider/rpm/syncer_test.go

@@ -0,0 +1,312 @@
+package rpm
+
+import (
+	"context"
+	"net/http"
+	"net/http/httptest"
+	"sync"
+	"testing"
+	"time"
+
+	"golang.org/x/time/rate"
+
+	"git.unkin.net/unkin/artifactapi/internal/provider"
+	"git.unkin.net/unkin/artifactapi/internal/testsupport"
+	"git.unkin.net/unkin/artifactapi/pkg/models"
+)
+
+// fakeSyncStore is an in-memory SyncStore: the metadata cache (via the embedded
+// fakeStore) plus the shared sync-state lease, whose claim mirrors the atomic
+// semantics of the real SQL (recency gate AND no live lease).
+type fakeSyncStore struct {
+	*fakeStore
+
+	mu         sync.Mutex
+	remotes    []models.Remote
+	leaseOwner map[string]string
+	leaseExp   map[string]time.Time
+	lastSynced map[string]time.Time
+	etags      map[string]string
+}
+
+func newFakeSyncStore() *fakeSyncStore {
+	return &fakeSyncStore{
+		fakeStore:  newFakeStore(),
+		leaseOwner: map[string]string{},
+		leaseExp:   map[string]time.Time{},
+		lastSynced: map[string]time.Time{},
+		etags:      map[string]string{},
+	}
+}
+
+func (f *fakeSyncStore) ListGitHubRPMRemotes(_ context.Context) ([]models.Remote, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	return append([]models.Remote(nil), f.remotes...), nil
+}
+
+func (f *fakeSyncStore) ClaimGitHubSyncLease(_ context.Context, name, owner string, freshness, lease time.Duration) (bool, string, error) {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	now := time.Now()
+	ls, hasLS := f.lastSynced[name]
+	exp, hasExp := f.leaseExp[name]
+	freshOK := !hasLS || now.Sub(ls) >= freshness
+	leaseOK := !hasExp || exp.Before(now)
+	if freshOK && leaseOK {
+		f.leaseOwner[name] = owner
+		f.leaseExp[name] = now.Add(lease)
+		return true, f.etags[name], nil
+	}
+	return false, "", nil
+}
+
+func (f *fakeSyncStore) ReleaseGitHubSyncLease(_ context.Context, name, owner, etag string, syncedAt time.Time) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if f.leaseOwner[name] != owner {
+		return nil
+	}
+	f.lastSynced[name] = syncedAt
+	f.etags[name] = etag
+	delete(f.leaseOwner, name)
+	delete(f.leaseExp, name)
+	return nil
+}
+
+func testSyncConfig() SyncConfig {
+	return SyncConfig{RatePerSec: 1000, Burst: 100, Workers: 1, PollInterval: time.Hour}
+}
+
+// (a) A 304 conditional response must derive nothing: no asset header GETs and
+// changed=false, so an unchanged repo is nearly free.
+func TestSyncerConditionalNotModifiedSkipsDerive(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	fx.etag = `"v1"`
+	p := newTestProvider()
+	store := newFakeStore()
+	remote := fx.remote()
+
+	etag1, changed, err := p.scanWithState(context.Background(), remote, store, "")
+	if err != nil {
+		t.Fatalf("first scan: %v", err)
+	}
+	if !changed || etag1 != `"v1"` {
+		t.Fatalf("first scan changed=%v etag=%q, want true and \"v1\"", changed, etag1)
+	}
+	priorRange := fx.rangeHit["demo-1.2-3.x86_64.rpm"]
+	if priorRange == 0 {
+		t.Fatal("first scan should have fetched the asset header")
+	}
+
+	etag2, changed2, err := p.scanWithState(context.Background(), remote, store, etag1)
+	if err != nil {
+		t.Fatalf("second scan: %v", err)
+	}
+	if changed2 {
+		t.Fatal("304 scan must report changed=false")
+	}
+	if etag2 != etag1 {
+		t.Fatalf("etag changed across 304: %q -> %q", etag1, etag2)
+	}
+	if fx.notModHit != 1 {
+		t.Fatalf("want exactly one 304 releases response, got %d", fx.notModHit)
+	}
+	if got := fx.rangeHit["demo-1.2-3.x86_64.rpm"]; got != priorRange {
+		t.Fatalf("304 scan re-fetched asset header: %d -> %d", priorRange, got)
+	}
+}
+
+// (b) On a real change, only the newly added asset is derived; assets already
+// cached are never re-fetched.
+func TestSyncerIncrementalDerivesOnlyNewAsset(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	fx.etag = `"v1"`
+	p := newTestProvider()
+	store := newFakeStore()
+	remote := fx.remote()
+
+	if _, _, err := p.scanWithState(context.Background(), remote, store, ""); err != nil {
+		t.Fatalf("first scan: %v", err)
+	}
+	demoRange := fx.rangeHit["demo-1.2-3.x86_64.rpm"]
+
+	// Add a new asset and bump the ETag so the conditional request returns 200.
+	fx.rpmBytes["other-9-9.aarch64.rpm"] = testsupport.MinimalRPM("other", "9", "9", "aarch64")
+	fx.etag = `"v2"`
+
+	if _, changed, err := p.scanWithState(context.Background(), remote, store, `"v1"`); err != nil || !changed {
+		t.Fatalf("second scan changed=%v err=%v", changed, err)
+	}
+
+	rows, _ := store.ListRPMMetadataEntries(context.Background(), remote.Name)
+	if len(rows) != 2 {
+		t.Fatalf("want 2 cached rows after incremental derive, got %d", len(rows))
+	}
+	if got := fx.rangeHit["demo-1.2-3.x86_64.rpm"]; got != demoRange {
+		t.Fatalf("already-cached asset was re-fetched: %d -> %d", demoRange, got)
+	}
+	if fx.rangeHit["other-9-9.aarch64.rpm"] == 0 {
+		t.Fatal("newly added asset was not derived")
+	}
+}
+
+// (c) The shared limiter caps the request rate: three gated releases calls at
+// one token per 120ms cannot complete faster than ~2 gaps.
+func TestRateLimiterCapsRequestRate(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	p := newTestProvider()
+	p.limiter = rate.NewLimiter(rate.Every(120*time.Millisecond), 1)
+	remote := fx.remote()
+
+	start := time.Now()
+	for i := 0; i < 3; i++ {
+		if _, _, _, err := p.fetchReleases(context.Background(), remote, ""); err != nil {
+			t.Fatalf("fetchReleases %d: %v", i, err)
+		}
+	}
+	if elapsed := time.Since(start); elapsed < 200*time.Millisecond {
+		t.Fatalf("rate limiter did not throttle: 3 calls took %v, want >= 200ms", elapsed)
+	}
+}
+
+// (d) Concurrent enqueues for the same remote coalesce to a single queued job.
+func TestSyncerEnqueueDedup(t *testing.T) {
+	store := newFakeSyncStore()
+	p := newTestProvider()
+	s := newSyncer(store, p, testSyncConfig())
+	remote := models.Remote{Name: "acme-rpm", PackageType: models.PackageGitHubRPM, MutableTTL: 3600}
+
+	var wg sync.WaitGroup
+	for i := 0; i < 10; i++ {
+		wg.Add(1)
+		go func() { defer wg.Done(); s.enqueue(remote, false) }()
+	}
+	wg.Wait()
+
+	if got := len(s.jobs); got != 1 {
+		t.Fatalf("want exactly 1 coalesced job, got %d", got)
+	}
+}
+
+// (e) Prime-on-create enqueues a prime job.
+func TestSyncerEnqueuePrime(t *testing.T) {
+	store := newFakeSyncStore()
+	p := newTestProvider()
+	s := newSyncer(store, p, testSyncConfig())
+	remote := models.Remote{Name: "acme-rpm", PackageType: models.PackageGitHubRPM, MutableTTL: 3600}
+
+	s.EnqueuePrime(remote)
+	select {
+	case job := <-s.jobs:
+		if !job.prime || job.remote.Name != "acme-rpm" {
+			t.Fatalf("bad prime job: %+v", job)
+		}
+	default:
+		t.Fatal("EnqueuePrime did not enqueue a job")
+	}
+}
+
+// (f) A held lease prevents a second replica from scanning: with the lease owned
+// by another replica, process claims nothing and makes zero GitHub calls.
+func TestSyncerLeasePreventsSecondReplica(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	fx.etag = `"v1"`
+	store := newFakeSyncStore()
+	p := newTestProvider()
+	s := newSyncer(store, p, testSyncConfig())
+	remote := fx.remote()
+
+	// Replica 1 holds the lease.
+	claimed, _, err := store.ClaimGitHubSyncLease(context.Background(), remote.Name, "replica-1", time.Duration(remote.MutableTTL)*time.Second, syncLeaseDuration)
+	if err != nil || !claimed {
+		t.Fatalf("replica-1 claim: claimed=%v err=%v", claimed, err)
+	}
+
+	// Replica 2 (this syncer) tries to process the same remote; it must skip.
+	s.process(context.Background(), syncJob{remote: remote})
+
+	if fx.releasesHit != 0 {
+		t.Fatalf("second replica scanned while lease held: %d releases calls", fx.releasesHit)
+	}
+	if rows, _ := store.ListRPMMetadataEntries(context.Background(), remote.Name); len(rows) != 0 {
+		t.Fatalf("second replica derived metadata while lease held: %d rows", len(rows))
+	}
+}
+
+// With the syncer wired and the cache empty, a repodata request enqueues a
+// prime and, when it has not landed within the bounded cold wait, returns a
+// retryable 503 rather than serving empty repodata (and without regressing the
+// detached-context serve).
+func TestServeRemoteColdStartReturns503(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	store := newFakeSyncStore()
+	p := newTestProvider()
+	p.coldWait = 300 * time.Millisecond
+	_ = newSyncer(store, p, testSyncConfig()) // binds p.syncer, but no workers running
+	remote := fx.remote()
+
+	rec := httptest.NewRecorder()
+	req := httptest.NewRequest(http.MethodGet, "/api/v1/remote/acme-rpm/repodata/repomd.xml", nil)
+	if !p.ServeRemote(rec, req, remote, "repodata/repomd.xml", "https://x", store) {
+		t.Fatal("ServeRemote did not handle repomd.xml")
+	}
+	if rec.Code != http.StatusServiceUnavailable {
+		t.Fatalf("cold empty cache must return 503, got %d", rec.Code)
+	}
+	if rec.Header().Get("Retry-After") == "" {
+		t.Fatal("503 should carry Retry-After")
+	}
+	// The prime was enqueued.
+	if got := len(p.syncer.jobs); got != 1 {
+		t.Fatalf("cold start did not enqueue a prime, jobs=%d", got)
+	}
+}
+
+// With the cache warm, the same request serves repodata immediately (no 503).
+func TestServeRemoteWarmCacheServesImmediately(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	store := newFakeSyncStore()
+	p := newTestProvider()
+	_ = newSyncer(store, p, testSyncConfig())
+	remote := fx.remote()
+
+	if err := p.scan(context.Background(), remote, store); err != nil {
+		t.Fatalf("warm scan: %v", err)
+	}
+	rec := httptest.NewRecorder()
+	req := httptest.NewRequest(http.MethodGet, "/api/v1/remote/acme-rpm/repodata/repomd.xml", nil)
+	if !p.ServeRemote(rec, req, remote, "repodata/repomd.xml", "https://x", store) {
+		t.Fatal("ServeRemote did not handle repomd.xml")
+	}
+	if rec.Code != http.StatusOK {
+		t.Fatalf("warm cache must serve 200, got %d body=%s", rec.Code, rec.Body.String())
+	}
+}
+
+// A prime job (freshness 0) runs even right after a sync, deriving metadata,
+// while a periodic job at the same moment is gated by the recency window.
+func TestSyncerPrimeBypassesRecencyPeriodicDoesNot(t *testing.T) {
+	fx := newGitHubFixture(t, true)
+	fx.etag = `"v1"`
+	store := newFakeSyncStore()
+	p := newTestProvider()
+	s := newSyncer(store, p, testSyncConfig())
+	remote := fx.remote()
+
+	var _ provider.RemoteMetadataStore = store
+
+	// Prime derives despite no prior sync.
+	s.process(context.Background(), syncJob{remote: remote, prime: true})
+	if rows, _ := store.ListRPMMetadataEntries(context.Background(), remote.Name); len(rows) != 1 {
+		t.Fatalf("prime did not derive: %d rows", len(rows))
+	}
+	releasesAfterPrime := fx.releasesHit
+
+	// A periodic job immediately after is gated by mutable_ttl recency: no new
+	// releases call.
+	s.process(context.Background(), syncJob{remote: remote, prime: false})
+	if fx.releasesHit != releasesAfterPrime {
+		t.Fatalf("periodic scan ran inside recency window: %d -> %d releases calls", releasesAfterPrime, fx.releasesHit)
+	}
+}

internal/server/server.go

@@ -19,6 +19,7 @@ import (
 	"git.unkin.net/unkin/artifactapi/internal/config"
 	"git.unkin.net/unkin/artifactapi/internal/database"
 	"git.unkin.net/unkin/artifactapi/internal/gc"
+	"git.unkin.net/unkin/artifactapi/internal/githubauth"
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/alpine"
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/docker"
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/generic"
@@ -27,7 +28,7 @@ import (
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/npm"
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/puppet"
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/pypi"
-	_ "git.unkin.net/unkin/artifactapi/internal/provider/rpm"
+	"git.unkin.net/unkin/artifactapi/internal/provider/rpm"
 	_ "git.unkin.net/unkin/artifactapi/internal/provider/terraform"
 	"git.unkin.net/unkin/artifactapi/internal/proxy"
 	"git.unkin.net/unkin/artifactapi/internal/storage"
@@ -47,6 +48,7 @@ type Server struct {
 	localHandler *v2.LocalHandler
 	tfRegistry   *tfregistry.Handler
 	gc           *gc.Collector
+	syncer       *rpm.Syncer
 }
 
 func New(cfg *config.Config, version string) (*Server, error) {
@@ -65,10 +67,35 @@ func New(cfg *config.Config, version string) (*Server, error) {
 		return nil, fmt.Errorf("s3: %w", err)
 	}
 
+	// Install the process-wide GitHub credential before any provider makes an
+	// outbound call. A misconfiguration (e.g. App id without a private key) fails
+	// closed here rather than silently falling back to anonymous. No credential
+	// configured is fine — requests stay anonymous.
+	ghCred, err := githubauth.New(githubauth.Options{
+		Token:          cfg.GitHubToken,
+		AppID:          cfg.GitHubAppID,
+		InstallationID: cfg.GitHubAppInstallationID,
+		PrivateKeyPEM:  cfg.GitHubAppPrivateKey,
+		PrivateKeyPath: cfg.GitHubAppPrivateKeyPath,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("github auth: %w", err)
+	}
+	githubauth.SetServer(ghCred)
+	if ghCred != nil {
+		slog.Info("github machine credential configured")
+	}
+
 	engine := proxy.NewEngine(db, redis, s3)
 	localHandler := v2.NewLocalHandler(db, s3)
 	virtEngine := virtual.NewEngine(db, engine)
 	collector := gc.New(db, s3, 1*time.Hour)
+	syncer := rpm.NewSyncer(db, rpm.SyncConfig{
+		RatePerSec:   cfg.GitHubSyncRatePerSec,
+		Burst:        cfg.GitHubSyncBurst,
+		Workers:      cfg.GitHubSyncWorkers,
+		PollInterval: time.Duration(cfg.GitHubSyncPollInterval) * time.Second,
+	})
 
 	// The terraform registry signs with a GPG key. A configured file wins (BYO
 	// key); otherwise artifactapi generates one on first start and persists it in
@@ -100,6 +127,7 @@ func New(cfg *config.Config, version string) (*Server, error) {
 		localHandler: localHandler,
 		tfRegistry:   tfRegistry,
 		gc:           collector,
+		syncer:       syncer,
 	}
 
 	s.router = s.routes()
@@ -129,7 +157,7 @@ func (s *Server) routes() chi.Router {
 	r.Mount("/api/v1", proxyHandler.Routes())
 	r.Mount("/v2", proxyHandler.DockerV2Routes())
 
-	remotesHandler := v2.NewRemotesHandler(s.db)
+	remotesHandler := v2.NewRemotesHandler(s.db, s.syncer)
 	virtualsHandler := v2.NewVirtualsHandler(s.db)
 	healthHandler := v2.NewHealthHandler(s.db, s.cache, s.store)
 	statsHandler := v2.NewStatsHandler(s.db)
@@ -196,6 +224,7 @@ func (s *Server) newHTTPServer() *http.Server {
 
 func (s *Server) Run(ctx context.Context) error {
 	go s.gc.Run(ctx)
+	go s.syncer.Run(ctx)
 
 	httpServer := s.newHTTPServer()
 
@@ -216,6 +245,7 @@ func (s *Server) Run(ctx context.Context) error {
 
 func (s *Server) RunOnListener(ctx context.Context, ln net.Listener) error {
 	go s.gc.Run(ctx)
+	go s.syncer.Run(ctx)
 
 	httpServer := s.newHTTPServer()