3 Commits

Author SHA1 Message Date
unkin-agent 73c0bfc670 deb/apk: make local repodata deterministic (#119)
ci/woodpecker/tag/docker Pipeline was successful
Part of #117. Local generated repodata must be byte-identical across the two no-affinity replicas and across every regeneration, so apt/apk never hit a checksum mismatch between an index's advertised hash and the bytes actually served. This does the deb+apk half (the rpm half landed in #118).

How:
- Derive the deb `Release` `Date:` from the newest persisted `created_at` (RFC1123Z, UTC) instead of `time.Now()`; carry `created_at` through the deb metadata SELECT and `DebMetadata`. An empty repo falls back to the Unix epoch. This also stops `Date:` running ahead of wall clock.
- Pin the apk `APKINDEX` tar header `ModTime` to the Unix epoch instead of the zero-value `time.Time`, so it is never wall-clock derived.
- Give both list queries a genuine total order by adding a `file_path` tiebreak (name/version/arch is not unique).
- Add guard tests: deb generators byte-identical across generations; the `Release` checksum/size matches the served `Packages`/`Packages.gz` bytes (the exact apt invariant); `Date:` pinned to `created_at`; apk index byte-identical and tar `ModTime` pinned to epoch.

Reviewed-on: #119
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-08-12 23:32:07 +10:00
unkin-agent 5fde0ee58e Add github_deb metadata-only package type (#112)
ci/woodpecker/tag/docker Pipeline was successful
## Why

This stacks the Debian/apt analog of `github_rpm` on top of the deb local+remote work (#111). It lets a GitHub repo's `.deb` release assets be consumed as a real apt repository without artifactapi ever precaching whole packages: it derives per-asset control metadata from a ranged prefix fetch, synthesizes a flat apt repo from the cache, and redirects the actual `.deb` downloads to a backend `releases_remote` (the generic github.com remote).

Base is `benvin/deb-local-remote` (stacked) to keep the diff atomic.

## How

- Adds `github_deb` to the package-type enum and validity map.
- Adds the `github_deb` provider mirroring `github_rpm`: `ServeRemote` serves `Packages`/`Packages.gz`/`Release`, returns 404 for `InRelease`/`Release.gpg` (unsigned, consumed via `[trusted=yes]`), and 302-redirects `*.deb` to `{proxyBaseURL}/api/v1/remote/{releases_remote}/{path}`; cold-start prime with a retryable 503.
- `deriveAsset` ranged-GETs the front of the `.deb` (an `ar` archive), locates and fully reads `control.tar.*`, and parses the control paragraph — doubling the range if the control member is truncated. The Packages `SHA256` comes from the GitHub asset `digest` when present, else a one-time full stream; `MD5sum` is left unset (apt verifies against SHA256 under `[trusted=yes]`).
- Adds a `github_deb` background Syncer (own worker pool, shared rate limiter, deduped queue) with per-remote DB-lease-gated scans so only one replica scans per window.
- Adds the `github_deb_sync_state` table plus `ListGitHubDebRemotes` / `ClaimGitHubDebSyncLease` / `ReleaseGitHubDebSyncLease` DB helpers, kept separate from the rpm ones.
- Primes `github_deb` remotes on create and runs the deb syncer alongside the rpm one; prime-on-create is routed by package type.
- Reuses the deb apt-index generators and control parser; the Packages generator now skips empty hash lines so a SHA256-only entry is valid.

## Notes / deviations

- **Filename convention:** the `Filename` stored in the Packages index is the **github-relative** asset path (same as rpm's `assetPath`), not `pool/<asset>`. This is required for the `.deb` 302 to `{releases_remote=github}/{path}` to resolve against github.com; it still matches the `*.deb` redirect rule.
- **GitHub client helpers** (releases pagination, ranged GET, auth headers) are duplicated into the deb package rather than shared, because the rpm equivalents are unexported in `package rpm` and the task requires not modifying the rpm provider.
- `go build`, `go vet`, `go mod tidy`, and `make test` (`-race`, incl. the Postgres lease integration tests) all pass; pre-commit clean.

Do not merge — for review.

---------

Co-authored-by: unkin-agent <unkin-agent@git.unkin.net>
Reviewed-on: #112
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-08-11 23:28:11 +10:00
unkin-agent 60f008debc Add deb (Debian/apt) local and remote repository support (#111)
Brings Debian/apt to artifactapi with feature parity to the existing rpm support (local + remote), so `.deb` packages can be hosted as a flat apt repo and a Debian/Ubuntu mirror can be cached through the proxy.

- Adds `deb` to the package-type enum and registers a new `internal/provider/deb` provider.
- Classifies `.deb` blobs immutable and the apt index surface (`Packages`, `Release`, `InRelease`, `dists/`, by-hash) mutable so the caching engine revalidates it.
- Parses the `.deb` in pure Go (ar archive to `control.tar.{gz,xz,zst}` to `./control`), storing the raw control stanza plus computed size/md5/sha256 as `deb_metadata`.
- Serves a flat apt repo (`deb [trusted=yes] .../ ./`): generates `Packages`, `Packages.gz` and an unsigned `Release` (returns 404 for `InRelease`/`Release.gpg`), mirroring rpm unsigned repodata / gpgcheck=0 trust model.
- Proxies a remote mirror via `UpstreamURL`/`ContentType`/`AuthHeaders` (HTTP Basic).
- Adds the `deb_metadata` table to `migrate()`, DB access methods, a `MinimalDeb` pure-Go fixture, unit tests, and a `dockere2e` `TestLocalDebRepo`.

---------

Co-authored-by: unkin-agent <unkin-agent@git.unkin.net>
Reviewed-on: #111
Co-authored-by: Unkin Agent <unkin-agent@unkin.net>
Co-committed-by: Unkin Agent <unkin-agent@unkin.net>
2026-08-11 23:21:08 +10:00