Add cache_repodata toggle for github_rpm remotes (#168)

github_rpm remotes regenerate repomd and all three gz files from every metadata row on each request, which is expensive for large repos under repeated dnf traffic.

- add a per-remote `cache_repodata` toggle (column, model, v2 API, docs)
- cache built repodata per replica, rebuilt via singleflight when the `count(*)`+`max(id)` row fingerprint moves
- store a build only if no newer build has landed; evict idle entries hourly
- map cached-path DB errors like the uncached path: 503 + Retry-After on cancel/deadline, generic 500 otherwise
- check `cacheEmpty` via the fingerprint for all github_rpm remotes, not only cached ones

Reviewed-on: #168
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
This commit was merged in pull request #168.
This commit is contained in:
2026-10-25 20:21:19 +11:00
committed by BenVincent
parent ee55ef3fcd
commit f1b78988e4
13 changed files with 648 additions and 26 deletions
+9
View File
@@ -72,6 +72,7 @@ resource "artifactapi_remote_github_rpm" "acme-tools" {
base_url = "https://api.github.com/repos/acme/tools" # the releases API root
releases_remote = "github" # backend for downloads
mutable_ttl = 3600 # release re-scan interval
cache_repodata = true # keep generated repodata in memory
# Optional: restrict which release assets become packages (regex on filename).
patterns = [".*\\.x86_64\\.rpm$", ".*\\.noarch\\.rpm$"]
@@ -84,3 +85,11 @@ resource "artifactapi_remote_github_rpm" "acme-tools" {
`dnf` config: `baseurl=https://artifactapi.example/api/v1/remote/acme-tools`.
The repo is multi-arch (no `$basearch` needed) — `dnf` selects matching packages
from the synthesized metadata.
### Repodata cache
Repodata is regenerated from the metadata rows on every request by default. Set
`cache_repodata = true` on the remote to keep the generated `repomd.xml` and its
`primary`/`filelists`/`other` files in memory per replica. Each request checks a
cheap row fingerprint (count + newest id) in the database and rebuilds only when
it moved, so every replica picks up a scan's changes on its next request.