unkin/artifactapi
3
0
Fork 0
Code Issues 5 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

Add Alpine/apk local repository support #114

Merged
benvin merged 1 commits from benvin/apk-local into master 2026-08-12 20:39:27 +10:00
Conversation 0 Commits 1 Files Changed 7
+917
unkin-agent commented 2026-08-12 00:59:55 +10:00
Member
Copy Link
Copy Source

Why

The alpine provider only supported remote (proxy) repositories, so there was no way to publish first-party .apk packages the way rpm-local and deb-local already allow. This extends the existing alpine provider into a real apk repository: uploaded .apk files are parsed in pure Go and a per-arch APKINDEX.tar.gz is generated on demand, at parity with rpm repodata and deb Packages generation. (The metadata-only github_alpine type is a separate follow-up and is not part of this PR.)

How

  • Implements LocalUploader / LocalIndexer / PostUploadHook / PostDeleteHook on the existing alpine provider, leaving the remote proxy methods (UpstreamURL/ContentType/AuthHeaders/RewriteResponse/Classify) intact.
  • Parses the .apk (up to three concatenated, independently gzipped tar streams) in pure Go: locates the control stream by its .PKGINFO member, reads the key = value fields, and computes the apk pull checksum C: = Q1 + base64(sha1(control gzip stream bytes)) — the sha1 of the second gzip member, not of the whole file.
  • Derives arch from .PKGINFO and records download size (S: blob size) and installed size (I: from .PKGINFO size).
  • Generates an unsigned per-arch APKINDEX.tar.gz = gzip(tar(APKINDEX)) filtered by requested arch (clients use --allow-untrusted, matching rpm gpgcheck=0 / deb [trusted=yes]), applying the same dot-segment normalization as deb so ./<arch>/APKINDEX.tar.gz resolves. Non-index / .apk paths return false so the generic file streamer serves the stored blob.
  • Adds AlpineMetadata plus separate AlpineMetadataStore / AlpineMetadataReader / AlpineMetadataDeleter interfaces (type-asserted from the generic hooks) so the shared rpm/deb metadata interfaces and their test doubles are untouched.
  • Adds the alpine_metadata table (keyed by repo_name + file_path, per-arch index) and its Insert/Delete/List DB methods.
  • Adds testsupport.MinimalApk, unit tests (.PKGINFO parse, Q1 checksum over the control stream, per-arch filtering, empty-field omission, ./ dot-segment handling, ValidateUpload accept/reject), and a dockere2e TestLocalAlpineIndex.

Consumption

/etc/apk/repositories line = <url>/api/v1/local/<name> (apk appends /<arch>/APKINDEX.tar.gz); apk update --allow-untrusted && apk add --allow-untrusted <pkg>. Packages live at /api/v1/local/<name>/<arch>/<file>.apk.

Verification

go build ./..., go vet ./... (incl. -tags dockere2e), go mod tidy (no change), make test (-race), and pre-commit run --all-files all pass.

## Why The alpine provider only supported remote (proxy) repositories, so there was no way to publish first-party `.apk` packages the way `rpm-local` and `deb-local` already allow. This extends the existing alpine provider into a real apk repository: uploaded `.apk` files are parsed in pure Go and a per-arch `APKINDEX.tar.gz` is generated on demand, at parity with rpm repodata and deb Packages generation. (The metadata-only `github_alpine` type is a separate follow-up and is not part of this PR.) ## How - Implements `LocalUploader` / `LocalIndexer` / `PostUploadHook` / `PostDeleteHook` on the existing `alpine` provider, leaving the remote proxy methods (`UpstreamURL`/`ContentType`/`AuthHeaders`/`RewriteResponse`/`Classify`) intact. - Parses the `.apk` (up to three concatenated, independently gzipped tar streams) in pure Go: locates the control stream by its `.PKGINFO` member, reads the `key = value` fields, and computes the apk pull checksum `C:` = `Q1` + base64(sha1(**control gzip stream bytes**)) — the sha1 of the second gzip member, not of the whole file. - Derives arch from `.PKGINFO` and records download size (`S:` blob size) and installed size (`I:` from `.PKGINFO size`). - Generates an **unsigned** per-arch `APKINDEX.tar.gz` = gzip(tar(`APKINDEX`)) filtered by requested arch (clients use `--allow-untrusted`, matching rpm `gpgcheck=0` / deb `[trusted=yes]`), applying the same dot-segment normalization as deb so `./<arch>/APKINDEX.tar.gz` resolves. Non-index / `.apk` paths return `false` so the generic file streamer serves the stored blob. - Adds `AlpineMetadata` plus **separate** `AlpineMetadataStore` / `AlpineMetadataReader` / `AlpineMetadataDeleter` interfaces (type-asserted from the generic hooks) so the shared rpm/deb metadata interfaces and their test doubles are untouched. - Adds the `alpine_metadata` table (keyed by `repo_name` + `file_path`, per-arch index) and its `Insert`/`Delete`/`List` DB methods. - Adds `testsupport.MinimalApk`, unit tests (`.PKGINFO` parse, Q1 checksum over the control stream, per-arch filtering, empty-field omission, `./` dot-segment handling, ValidateUpload accept/reject), and a `dockere2e` `TestLocalAlpineIndex`. ## Consumption `/etc/apk/repositories` line = `<url>/api/v1/local/<name>` (apk appends `/<arch>/APKINDEX.tar.gz`); `apk update --allow-untrusted && apk add --allow-untrusted <pkg>`. Packages live at `/api/v1/local/<name>/<arch>/<file>.apk`. ## Verification `go build ./...`, `go vet ./...` (incl. `-tags dockere2e`), `go mod tidy` (no change), `make test` (`-race`), and `pre-commit run --all-files` all pass.
unkin-agent added 1 commit 2026-08-12 00:59:55 +10:00
Add Alpine/apk local repository support
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/build Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
58a24a15dd 
The alpine provider hosted only remote (proxy) repos; there was no way to
publish first-party .apk packages the way rpm-local and deb-local already
allow. This extends the existing alpine provider into a real apk repository:
uploaded .apk files are parsed in pure Go and a per-arch APKINDEX.tar.gz is
generated on demand, at parity with rpm repodata and deb Packages generation.

- Implement LocalUploader/LocalIndexer/PostUploadHook/PostDeleteHook on the
  alpine provider, keeping the remote proxy methods intact.
- Parse .apk (concatenated gzipped tar streams) in pure Go: read .PKGINFO from
  the control stream and compute the apk pull checksum C: = Q1+base64(sha1) over
  the raw control gzip stream (not the whole file).
- Generate an unsigned per-arch APKINDEX.tar.gz (clients use --allow-untrusted),
  applying the same dot-segment normalization as deb for ./<arch>/... requests.
- Add AlpineMetadata plus separate Alpine store/reader/deleter interfaces so the
  rpm/deb metadata interfaces are not widened.
- Add the alpine_metadata table and its Insert/Delete/List DB methods.
- Add testsupport.MinimalApk plus unit tests (parse, Q1 checksum, per-arch
  filtering, dot-segment handling, validate) and a dockere2e index test.
benvin merged commit 7f77666709 into master 2026-08-12 20:39:27 +10:00
benvin deleted branch benvin/apk-local 2026-08-12 20:39:27 +10:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/artifactapi#114
Delete Branch "benvin/apk-local"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?