Commit Graph

3 Commits

Author SHA1 Message Date
unkin-agent 7fa5434dbc Add deb virtual repositories (#138)
ci/woodpecker/tag/docker Pipeline was successful
Virtual repos merge helm, pypi, rpm, npm, gems, ansible, goproxy and puppet but not deb, so several apt repos (e.g. github_deb remotes plus a local deb) cannot be served as one apt source.

- merge member Packages for flat (`./`) and `dists/<suite>` layouts; member order wins duplicate (Package, Version, Architecture)
- generate an unsigned Release with Acquire-By-Hash, serving indexes by name and by SHA256 from both retained generations
- prefix Filename with the member name and 302 downloads to the member route
- cache merges per suite on the shared virtual cache, capped at 32 suites per virtual (LRU)
- add docs/deb/virtual.md, unit tests and dockerised apt e2e for flat and suite layouts

Reviewed-on: #138
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-24 16:21:33 +11:00
unkin-agent 0150b5ebb4 Share the virtual merge cache across index types (#147)
The rpm virtual's merge cache, singleflight and two-generation retention are hard-wired to RPMRepo, so other merged index types (deb next) can't reuse them.

- add a mergedRepo interface (index + referenced files) implemented by RPMRepo
- key the merge cache by type-scoped string and serve files from both retained generations via cachedFile
- extract parallel fail-closed member fetching into a generic fetchMembers
- add tests for generation retention and per-key isolation

Reviewed-on: #147
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-17 13:28:15 +11:00
unkin-agent 1837f6ef8c Add rpm virtual repositories (#131)
ci/woodpecker/tag/docker Pipeline failed
Virtual repos only merge helm and pypi, so several rpm repos (e.g. many github_rpm remotes) cannot be served as one yum repo.

- merge member primary/filelists/other into one repodata set; member order wins duplicate NEVRAs
- read member repodata through local, github_rpm and proxied remote paths
- prefix package locations (incl. xml:base under a member upstream) with the member name and 302 them to the member route
- reject absolute and dot-segment member paths; escape the redirect and keep its query
- return 502 when any member's repodata is unavailable
- reuse each virtual's merge for 60s behind singleflight; serve data files from its current and previous merge (per replica)
- add merger/engine unit tests and a dockerised dnf e2e case

Reviewed-on: #131
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-09 22:07:20 +11:00