rpm: make local repodata deterministic (fixes #117) #118

2026-08-12T22:49:34+10:00

unkin-agent commented

2026-08-12 22:49:34 +10:00

Local RPM repos regenerated repomd.xml on every request and advertised a primary.xml.gz sha256 that drifted every second, because time.Now().Unix() was embedded inside the gzipped primary.xml (and in repomd <revision>/<timestamp>). The advertised hash therefore never matched the content-addressed <sha256>-primary.xml.gz bytes a second later or on the other replica, so dnf failed with a checksum mismatch. Part of #117.

How:

Derives <time file=> in primary.xml from the persisted rpm_metadata.created_at instead of the wall clock; unset timestamps collapse to a fixed 0.
Derives repomd <revision>/<timestamp> from the newest package upload time, so repomd.xml is byte-identical across replicas and requests.
Adds file_path as a total-order tiebreak to the metadata ORDER BY.
Pins the gzip header (OS: 255) so compressed bytes depend only on the payload.
Adds regression tests: generators are byte-identical across two runs, and the sha256 in repomd.xml equals the sha256 of the bytes each serve* handler returns.

Local RPM repos regenerated `repomd.xml` on every request and advertised a `primary.xml.gz` sha256 that drifted every second, because `time.Now().Unix()` was embedded inside the gzipped `primary.xml` (and in `repomd` `<revision>`/`<timestamp>`). The advertised hash therefore never matched the content-addressed `<sha256>-primary.xml.gz` bytes a second later or on the other replica, so `dnf` failed with a checksum mismatch. Part of #117. How: - Derives `<time file=>` in `primary.xml` from the persisted `rpm_metadata.created_at` instead of the wall clock; unset timestamps collapse to a fixed `0`. - Derives `repomd` `<revision>`/`<timestamp>` from the newest package upload time, so `repomd.xml` is byte-identical across replicas and requests. - Adds `file_path` as a total-order tiebreak to the metadata `ORDER BY`. - Pins the gzip header (`OS: 255`) so compressed bytes depend only on the payload. - Adds regression tests: generators are byte-identical across two runs, and the sha256 in `repomd.xml` equals the sha256 of the bytes each `serve*` handler returns.

unkin-agent added 1 commit 2026-08-12 22:49:34 +10:00

rpm: make local repodata deterministic

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

d4dfc4421a

repomd.xml regenerates per request and advertised a primary.xml.gz sha256
that drifted every second (time.Now() embedded in the gzipped primary.xml),
so dnf failed the checksum on the content-addressed <sha256>-primary.xml.gz.

Fixes #117.

- Derive <time file=> from the persisted rpm_metadata.created_at instead of
  time.Now(); unset timestamps collapse to a fixed 0.
- Derive repomd <revision>/<timestamp> from the newest package upload time so
  repomd.xml is byte-identical across replicas and requests.
- Add a total-order file_path tiebreak to the metadata ORDER BY.
- Pin the gzip header so compressed bytes depend only on the payload.

benvin merged commit a8aa0c231b into master

2026-08-12 23:14:52 +10:00

benvin deleted branch benvin/rpm-repodata-deterministic

2026-08-12 23:14:52 +10:00

unkin-agent referenced this pull request

2026-08-12 23:28:14 +10:00

deb/apk: make local repodata deterministic #119

benvin referenced this issue from a commit

2026-08-12 23:32:08 +10:00

deb/apk: make local repodata deterministic (#119)

unkin-agent referenced this pull request

2026-08-12 23:33:23 +10:00