feat: add github_rpm metadata-only remote serving GitHub releases as a yum repo
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful

Publishing RPMs to GitHub releases is common, but consuming them with dnf
requires repodata that GitHub does not provide, and mirroring every package
into a local repo wastes storage and staleness-tracking on artifacts that
already have a durable home. Expose GitHub releases as a first-class RPM source
that synthesizes repodata on the fly and never precaches the packages.

Add a `github_rpm` remote package type backed by a metadata-only provider:

- Introduce a `RemoteServer` interception hook (the remote-side analog of
  `LocalIndexer`): `handleProxy` lets a provider fully answer a request before
  the byte-proxy engine, passing the request-derived proxy base URL and the DB
  as a `RemoteMetadataStore`.
- Scan a repo's releases via the GitHub API (`base_url` = the releases API
  root) for `.rpm` assets, filtered by the remote's `patterns` (regex on asset
  filename) and reuse the existing local-rpm repodata generators to emit
  `repomd.xml`/`primary`/`filelists`/`other`.
- Derive per-asset metadata without precaching: fetch only the RPM header via a
  ranged GET (retrying with a larger range on a truncated-header parse) for
  NEVRA, requires/provides/conflicts/obsoletes and files; take the sha256 from
  the GitHub asset `digest` when present, else compute it once by streaming.
- Cache derived metadata in `rpm_metadata` keyed by asset path; re-scan no more
  often than `mutable_ttl`, pruning assets that disappear upstream.
- Serve each package's `<location>` as the github-relative download path so the
  client comes back to this remote, which 302-redirects to the `releases_remote`
  (an existing generic github.com remote) that streams the actual bytes.

Reuse the existing `releases_remote` field as the redirect target — it already
carries exactly this "downloads served by remote X" semantic end to end.

Extend the shared RPM metadata model with conflicts/obsoletes (JSONB columns,
added idempotently) so both local and github_rpm repodata resolve upgrades and
conflicts; the local upload path now records them too.

Tests cover header-range parsing with the retry loop, digest-vs-computed
checksum selection, repodata synthesis with the redirect-able location href,
the 302 redirect path, asset pattern filtering, and stale-asset pruning.
This commit is contained in:
2026-08-10 09:35:22 +10:00
parent eee8ee1c31
commit b727b990a2
10 changed files with 964 additions and 5 deletions
+20
View File
@@ -63,6 +63,24 @@ type MetadataStore interface {
InsertRPMMetadata(ctx context.Context, meta *RPMMetadata) error
}
// RemoteServer lets a remote provider fully answer a request itself instead of
// going through the byte-proxy engine. It is the remote-side analog of
// LocalIndexer: a metadata-only remote (e.g. github_rpm) uses it to synthesize
// repodata from derived per-asset metadata and to redirect package downloads to
// a backend remote, without ever precaching the packages. Returning false lets
// the normal proxy path take over.
type RemoteServer interface {
ServeRemote(w http.ResponseWriter, r *http.Request, remote models.Remote, path, proxyBaseURL string, store RemoteMetadataStore) bool
}
// RemoteMetadataStore is the persistence surface a RemoteServer needs to cache
// and read the metadata it derives per upstream asset. *database.DB satisfies it.
type RemoteMetadataStore interface {
RPMMetadataReader
MetadataStore
MetadataDeleter
}
type MetadataDeleter interface {
DeleteRPMMetadata(ctx context.Context, repoName, filePath string) error
}
@@ -93,6 +111,8 @@ type RPMMetadata struct {
Packager string
Requires []RPMDep
Provides []RPMDep
Conflicts []RPMDep
Obsoletes []RPMDep
Files []RPMFile
Changelogs []RPMChangelog
}