artifactapi

unkin/artifactapi

Fork 0

Author	SHA1	Message	Date
unkin-agent	d1c3c2fb6c	Fix github_alpine .apk redirect to resolve stored FilePath ci/woodpecker/pr/build Pipeline was successful Details ci/woodpecker/pr/test Pipeline was successful Details ci/woodpecker/pr/pre-commit Pipeline was successful Details apk reconstructs the download URL itself as <arch>/<name>-<version>.apk because APKINDEX carries no filename field (unlike rpm's <location> or deb's Filename:). ServeRemote forwarded that synthesized path verbatim into the releases_remote redirect, pointing at a nonexistent, allowlist-denied github.com path (404/403). Look up the cached metadata row by arch plus the full reconstructed filename (no hyphen-split, so -rN suffixes are preserved) and redirect to the stored github-relative FilePath. Unknown packages now 404 instead of redirecting to a bad path.	2026-08-12 20:40:19 +10:00
unkin-agent	aa96c8af70	Add github_alpine metadata-only package type github_alpine is the Alpine/apk analog of github_deb/github_rpm: a metadata-only remote that scans a GitHub repo's releases for .apk assets, derives each package's .PKGINFO via a ranged prefix fetch (never downloading whole packages), synthesizes a per-arch APKINDEX.tar.gz from that cached metadata, and 302-redirects .apk downloads to a backend releases_remote. It stacks on the apk-local work, reusing the alpine provider's APKINDEX generator, .PKGINFO parser, Q1 checksum, and AlpineMetadata store. - pkg/models: add PackageGitHubAlpine to the enum + validators - internal/provider/alpine/github.go: the github_alpine provider (ServeRemote per-arch index + .apk redirect, cold-start 503, scanWithState incremental derive, ranged .PKGINFO prefix fetch with range-doubling on truncation) - internal/provider/alpine/syncer.go: parallel background Syncer (worker pool, shared limiter, deduped queue, DB lease) - internal/database/alpine_github_sync.go + github_alpine_sync_state table: remote enumeration + per-remote sync lease - internal/api/v2/remotes.go: primed on create via the shared Primer map - internal/server/server.go: construct + Run the alpine syncer, register it in the Primer map - tests mirror the deb github_test/syncer_test (scan/diff/prune, ranged .PKGINFO parse, per-arch ServeRemote routing, .apk 302, DB lease)	2026-08-12 20:40:19 +10:00
unkin-agent	7f77666709	Add Alpine/apk local repository support (#114 ) ## Why The alpine provider only supported remote (proxy) repositories, so there was no way to publish first-party `.apk` packages the way `rpm-local` and `deb-local` already allow. This extends the existing alpine provider into a real apk repository: uploaded `.apk` files are parsed in pure Go and a per-arch `APKINDEX.tar.gz` is generated on demand, at parity with rpm repodata and deb Packages generation. (The metadata-only `github_alpine` type is a separate follow-up and is not part of this PR.) ## How - Implements `LocalUploader` / `LocalIndexer` / `PostUploadHook` / `PostDeleteHook` on the existing `alpine` provider, leaving the remote proxy methods (`UpstreamURL`/`ContentType`/`AuthHeaders`/`RewriteResponse`/`Classify`) intact. - Parses the `.apk` (up to three concatenated, independently gzipped tar streams) in pure Go: locates the control stream by its `.PKGINFO` member, reads the `key = value` fields, and computes the apk pull checksum `C:` = `Q1` + base64(sha1(control gzip stream bytes)) — the sha1 of the second gzip member, not of the whole file. - Derives arch from `.PKGINFO` and records download size (`S:` blob size) and installed size (`I:` from `.PKGINFO size`). - Generates an unsigned per-arch `APKINDEX.tar.gz` = gzip(tar(`APKINDEX`)) filtered by requested arch (clients use `--allow-untrusted`, matching rpm `gpgcheck=0` / deb `[trusted=yes]`), applying the same dot-segment normalization as deb so `./<arch>/APKINDEX.tar.gz` resolves. Non-index / `.apk` paths return `false` so the generic file streamer serves the stored blob. - Adds `AlpineMetadata` plus separate `AlpineMetadataStore` / `AlpineMetadataReader` / `AlpineMetadataDeleter` interfaces (type-asserted from the generic hooks) so the shared rpm/deb metadata interfaces and their test doubles are untouched. - Adds the `alpine_metadata` table (keyed by `repo_name` + `file_path`, per-arch index) and its `Insert`/`Delete`/`List` DB methods. - Adds `testsupport.MinimalApk`, unit tests (`.PKGINFO` parse, Q1 checksum over the control stream, per-arch filtering, empty-field omission, `./` dot-segment handling, ValidateUpload accept/reject), and a `dockere2e` `TestLocalAlpineIndex`. ## Consumption `/etc/apk/repositories` line = `<url>/api/v1/local/<name>` (apk appends `/<arch>/APKINDEX.tar.gz`); `apk update --allow-untrusted && apk add --allow-untrusted <pkg>`. Packages live at `/api/v1/local/<name>/<arch>/<file>.apk`. ## Verification `go build ./...`, `go vet ./...` (incl. `-tags dockere2e`), `go mod tidy` (no change), `make test` (`-race`), and `pre-commit run --all-files` all pass. Reviewed-on: #114 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net>	2026-08-12 20:39:27 +10:00

Author

SHA1

Message

Date

unkin-agent

d1c3c2fb6c

Fix github_alpine .apk redirect to resolve stored FilePath

ci/woodpecker/pr/build Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

apk reconstructs the download URL itself as <arch>/<name>-<version>.apk
because APKINDEX carries no filename field (unlike rpm's <location> or
deb's Filename:). ServeRemote forwarded that synthesized path verbatim
into the releases_remote redirect, pointing at a nonexistent,
allowlist-denied github.com path (404/403).

Look up the cached metadata row by arch plus the full reconstructed
filename (no hyphen-split, so -rN suffixes are preserved) and redirect
to the stored github-relative FilePath. Unknown packages now 404 instead
of redirecting to a bad path.

2026-08-12 20:40:19 +10:00

unkin-agent

aa96c8af70

Add github_alpine metadata-only package type

github_alpine is the Alpine/apk analog of github_deb/github_rpm: a
metadata-only remote that scans a GitHub repo's releases for .apk assets,
derives each package's .PKGINFO via a ranged prefix fetch (never
downloading whole packages), synthesizes a per-arch APKINDEX.tar.gz from
that cached metadata, and 302-redirects .apk downloads to a backend
releases_remote. It stacks on the apk-local work, reusing the alpine
provider's APKINDEX generator, .PKGINFO parser, Q1 checksum, and
AlpineMetadata store.

- pkg/models: add PackageGitHubAlpine to the enum + validators
- internal/provider/alpine/github.go: the github_alpine provider
  (ServeRemote per-arch index + .apk redirect, cold-start 503,
  scanWithState incremental derive, ranged .PKGINFO prefix fetch with
  range-doubling on truncation)
- internal/provider/alpine/syncer.go: parallel background Syncer
  (worker pool, shared limiter, deduped queue, DB lease)
- internal/database/alpine_github_sync.go + github_alpine_sync_state
  table: remote enumeration + per-remote sync lease
- internal/api/v2/remotes.go: primed on create via the shared Primer map
- internal/server/server.go: construct + Run the alpine syncer, register
  it in the Primer map
- tests mirror the deb github_test/syncer_test (scan/diff/prune, ranged
  .PKGINFO parse, per-arch ServeRemote routing, .apk 302, DB lease)

2026-08-12 20:40:19 +10:00

unkin-agent

7f77666709

Add Alpine/apk local repository support (#114 )

## Why

The alpine provider only supported remote (proxy) repositories, so there was no way to publish first-party `.apk` packages the way `rpm-local` and `deb-local` already allow. This extends the existing alpine provider into a real apk repository: uploaded `.apk` files are parsed in pure Go and a per-arch `APKINDEX.tar.gz` is generated on demand, at parity with rpm repodata and deb Packages generation. (The metadata-only `github_alpine` type is a separate follow-up and is not part of this PR.)

## How

- Implements `LocalUploader` / `LocalIndexer` / `PostUploadHook` / `PostDeleteHook` on the existing `alpine` provider, leaving the remote proxy methods (`UpstreamURL`/`ContentType`/`AuthHeaders`/`RewriteResponse`/`Classify`) intact.
- Parses the `.apk` (up to three concatenated, independently gzipped tar streams) in pure Go: locates the control stream by its `.PKGINFO` member, reads the `key = value` fields, and computes the apk pull checksum `C:` = `Q1` + base64(sha1(**control gzip stream bytes**)) — the sha1 of the second gzip member, not of the whole file.
- Derives arch from `.PKGINFO` and records download size (`S:` blob size) and installed size (`I:` from `.PKGINFO size`).
- Generates an **unsigned** per-arch `APKINDEX.tar.gz` = gzip(tar(`APKINDEX`)) filtered by requested arch (clients use `--allow-untrusted`, matching rpm `gpgcheck=0` / deb `[trusted=yes]`), applying the same dot-segment normalization as deb so `./<arch>/APKINDEX.tar.gz` resolves. Non-index / `.apk` paths return `false` so the generic file streamer serves the stored blob.
- Adds `AlpineMetadata` plus **separate** `AlpineMetadataStore` / `AlpineMetadataReader` / `AlpineMetadataDeleter` interfaces (type-asserted from the generic hooks) so the shared rpm/deb metadata interfaces and their test doubles are untouched.
- Adds the `alpine_metadata` table (keyed by `repo_name` + `file_path`, per-arch index) and its `Insert`/`Delete`/`List` DB methods.
- Adds `testsupport.MinimalApk`, unit tests (`.PKGINFO` parse, Q1 checksum over the control stream, per-arch filtering, empty-field omission, `./` dot-segment handling, ValidateUpload accept/reject), and a `dockere2e` `TestLocalAlpineIndex`.

## Consumption

`/etc/apk/repositories` line = `<url>/api/v1/local/<name>` (apk appends `/<arch>/APKINDEX.tar.gz`); `apk update --allow-untrusted && apk add --allow-untrusted <pkg>`. Packages live at `/api/v1/local/<name>/<arch>/<file>.apk`.

## Verification

`go build ./...`, `go vet ./...` (incl. `-tags dockere2e`), `go mod tidy` (no change), `make test` (`-race`), and `pre-commit run --all-files` all pass.

Reviewed-on: #114
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>

2026-08-12 20:39:27 +10:00

Compare commits

3 Commits

6c6ad3066e .. d1c3c2fb6c

Diff Content Not Available

Compare commits

3 Commits 6c6ad3066e .. d1c3c2fb6c

Diff Content Not Available

3 Commits

6c6ad3066e .. d1c3c2fb6c