A cold `dnf makecache` against a github_rpm remote with many release
assets 500s on the first request: ServeRemote derives metadata for every
asset synchronously on the inbound request context, so once dnf hits its
makecache timeout and disconnects the canceled request context both
aborts the in-flight derive and poisons the subsequent repodata DB read,
which surfaces as HTTP 500. It only "works" on a lucky client retry that
finds the partially-populated cache fresh.
- detach the release scan to a background, timeout-bounded context so a
client cancel can neither abort the shared derive nor cancel the read
- single-flight the scan per remote so concurrent requests never launch
duplicate derives
- serve the current cache immediately when it is non-empty and derive in
the background; only a completely empty cache blocks on a bounded first
scan
- serve repodata on a context detached from the request, and treat a
canceled/deadline-exceeded metadata read as a retryable 503 instead of
a hard 500
Publishing RPMs to GitHub releases is common, but consuming them with dnf
requires repodata that GitHub does not provide, and mirroring every package
into a local repo wastes storage and staleness-tracking on artifacts that
already have a durable home. Expose GitHub releases as a first-class RPM source
that synthesizes repodata on the fly and never precaches the packages.
Add a `github_rpm` remote package type backed by a metadata-only provider:
- Introduce a `RemoteServer` interception hook (the remote-side analog of
`LocalIndexer`): `handleProxy` lets a provider fully answer a request before
the byte-proxy engine, passing the request-derived proxy base URL and the DB
as a `RemoteMetadataStore`.
- Scan a repo's releases via the GitHub API (`base_url` = the releases API
root) for `.rpm` assets, filtered by the remote's `patterns` (regex on asset
filename) and reuse the existing local-rpm repodata generators to emit
`repomd.xml`/`primary`/`filelists`/`other`.
- Derive per-asset metadata without precaching: fetch only the RPM header via a
ranged GET (retrying with a larger range on a truncated-header parse) for
NEVRA, requires/provides/conflicts/obsoletes and files; take the sha256 from
the GitHub asset `digest` when present, else compute it once by streaming.
- Cache derived metadata in `rpm_metadata` keyed by asset path; re-scan no more
often than `mutable_ttl`, pruning assets that disappear upstream.
- Serve each package's `<location>` as the github-relative download path so the
client comes back to this remote, which 302-redirects to the `releases_remote`
(an existing generic github.com remote) that streams the actual bytes.
Reuse the existing `releases_remote` field as the redirect target — it already
carries exactly this "downloads served by remote X" semantic end to end.
Extend the shared RPM metadata model with conflicts/obsoletes (JSONB columns,
added idempotently) so both local and github_rpm repodata resolve upgrades and
conflicts; the local upload path now records them too.
Tests cover header-range parsing with the retry loop, digest-vs-computed
checksum selection, repodata synthesis with the redirect-able location href,
the 302 redirect path, asset pattern filtering, and stale-asset pruning.
unkinben