unkin/artifactapi
3
0
Fork 0
Code Issues 5 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

feat: server-level GitHub machine credential for authenticated requests #109

Merged
benvin merged 1 commits from benvin/github-auth into master 2026-08-10 21:42:39 +10:00
Conversation 0 Commits 1 Files Changed 12
+968 -12
unkinben commented 2026-08-10 21:29:03 +10:00
Owner
Copy Link
Copy Source

Why

Anonymous GitHub is capped at 60 requests/hour and cannot read private repositories. A machine credential usable by a free (non-enterprise) account is needed to lift the request budget to ~5000/hr and to read private-repo release assets.

Builds on the background syncer (#108, now merged to master); this diff is the auth changes only.

How

  • Add internal/githubauth: a process-wide GitHub credential delivered via env/secret, applied by default to every outbound GitHub request (releases scan, ranged asset-header GETs, and the generic-github byte proxy for private assets).
  • Support two modes:
    • PAT — GITHUB_TOKEN sent as Authorization: Bearer <token>.
    • GitHub App — GITHUB_APP_ID + GITHUB_APP_INSTALLATION_ID + private key (GITHUB_APP_PRIVATE_KEY inline PEM or GITHUB_APP_PRIVATE_KEY_PATH). Mint a short-lived RS256 JWT with stdlib crypto/rsa (no new dependency), exchange it at POST /app/installations/{id}/access_tokens for a ~1h installation token, cache it, and single-flight a refresh a few minutes before expiry.
  • Inject at the two GitHub call paths: the rpm github provider header builder (releases + ranged fetches) and the generic provider AuthHeaders (byte proxy, github.com hosts only; the pre-signed objects.githubusercontent.com redirect deliberately gets no Authorization).
  • Honor precedence: a remote's own username/password overrides the server credential; no credential configured stays anonymous (current behavior).
  • Fail closed at startup on partial App configuration (e.g. App id without a private key); a token-and-App conflict is also rejected.
  • Never persist the credential to the DB, return it from an API, or log it (token-exchange failures never echo the response body).
  • Read config via the existing getenv convention; document PAT vs App setup, the free-account fine-grained PAT scopes (Contents:read + Metadata:read), precedence, and the rate-limit implication.

Rate limit

Authenticated requests share the syncer's single global limiter — no second limiter is added. A token raises the effective GitHub ceiling (~5000/hr vs ~60/hr), so the limiter defaults stay safe.

Tests

internal/githubauth and internal/provider/{rpm,generic}:

  • PAT attaches the correct Authorization header to releases + asset-header requests.
  • App mints a valid RS256 JWT (verified against the app public key), exchanges it at a mocked endpoint, reuses the cached token without re-exchanging, refreshes near expiry, and single-flights concurrent callers.
  • Per-remote credential overrides the server credential (rpm + generic).
  • No credential → no Authorization header, requests still succeed anonymously.
  • ETag/304 flow still works with auth attached.
  • The credential does not appear in a remote's serialized JSON.
  • Config validation: no-config is anonymous; partial App config and token/App conflict both error.

Verified fail-before/pass-after for the injection tests. gofmt -l, go build ./..., go vet ./..., go test ./... all clean (26 packages).

## Why Anonymous GitHub is capped at 60 requests/hour and cannot read private repositories. A machine credential usable by a free (non-enterprise) account is needed to lift the request budget to ~5000/hr and to read private-repo release assets. Builds on the background syncer (#108, now merged to `master`); this diff is the auth changes only. ## How - Add `internal/githubauth`: a process-wide GitHub credential delivered via env/secret, applied by default to every outbound GitHub request (releases scan, ranged asset-header GETs, and the generic-github byte proxy for private assets). - Support two modes: - **PAT** — `GITHUB_TOKEN` sent as `Authorization: Bearer <token>`. - **GitHub App** — `GITHUB_APP_ID` + `GITHUB_APP_INSTALLATION_ID` + private key (`GITHUB_APP_PRIVATE_KEY` inline PEM or `GITHUB_APP_PRIVATE_KEY_PATH`). Mint a short-lived RS256 JWT with stdlib `crypto/rsa` (no new dependency), exchange it at `POST /app/installations/{id}/access_tokens` for a ~1h installation token, cache it, and single-flight a refresh a few minutes before expiry. - Inject at the two GitHub call paths: the rpm github provider header builder (releases + ranged fetches) and the generic provider `AuthHeaders` (byte proxy, github.com hosts only; the pre-signed `objects.githubusercontent.com` redirect deliberately gets no Authorization). - Honor precedence: a remote's own `username`/`password` overrides the server credential; no credential configured stays anonymous (current behavior). - Fail closed at startup on partial App configuration (e.g. App id without a private key); a token-and-App conflict is also rejected. - Never persist the credential to the DB, return it from an API, or log it (token-exchange failures never echo the response body). - Read config via the existing `getenv` convention; document PAT vs App setup, the free-account fine-grained PAT scopes (Contents:read + Metadata:read), precedence, and the rate-limit implication. ## Rate limit Authenticated requests share the syncer's single global limiter — no second limiter is added. A token raises the effective GitHub ceiling (~5000/hr vs ~60/hr), so the limiter defaults stay safe. ## Tests `internal/githubauth` and `internal/provider/{rpm,generic}`: - PAT attaches the correct `Authorization` header to releases + asset-header requests. - App mints a valid RS256 JWT (verified against the app public key), exchanges it at a mocked endpoint, reuses the cached token without re-exchanging, refreshes near expiry, and single-flights concurrent callers. - Per-remote credential overrides the server credential (rpm + generic). - No credential → no `Authorization` header, requests still succeed anonymously. - ETag/304 flow still works with auth attached. - The credential does not appear in a remote's serialized JSON. - Config validation: no-config is anonymous; partial App config and token/App conflict both error. Verified fail-before/pass-after for the injection tests. `gofmt -l`, `go build ./...`, `go vet ./...`, `go test ./...` all clean (26 packages).
benvin changed target branch from benvin/github-rpm-syncer to master 2026-08-10 21:31:25 +10:00
unkinben added 1 commit 2026-08-10 21:33:06 +10:00
feat: server-level GitHub machine credential for authenticated requests
ci/woodpecker/pr/build Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
8ced48901f 
Anonymous GitHub is capped at 60 requests/hour and cannot read private
repositories, so a machine credential usable by a free (non-enterprise)
account is needed to lift the request budget and reach private release
assets.

- Add internal/githubauth: a process-wide credential delivered via env/secret,
  applied by default to every outbound GitHub request.
- Support two modes: a Personal Access Token sent as `Authorization: Bearer`,
  and a GitHub App that mints a short-lived RS256 JWT (stdlib crypto, no new
  dependency), exchanges it for a ~1h installation token, caches it, and
  single-flights a refresh a few minutes before expiry.
- Inject the credential at the two GitHub call paths: the rpm github provider
  (releases scan + ranged asset-header GETs) and the generic byte proxy
  (private release-asset downloads for github.com hosts).
- Honor precedence: a remote's own username/password overrides the server
  credential; no credential configured stays anonymous.
- Fail closed at startup on partial App configuration; never persist the
  credential to the DB, return it from an API, or log it.
- Read GITHUB_TOKEN / GITHUB_APP_ID / GITHUB_APP_INSTALLATION_ID /
  GITHUB_APP_PRIVATE_KEY[_PATH] via the existing getenv convention.
- Document PAT vs App setup, the free-account fine-grained PAT scopes
  (Contents:read + Metadata:read), precedence, and the rate-limit implication.
unkinben force-pushed benvin/github-auth from 9b94f18d7a to 8ced48901f 2026-08-10 21:33:06 +10:00 Compare
benvin merged commit 109ba2ce27 into master 2026-08-10 21:42:39 +10:00
benvin deleted branch benvin/github-auth 2026-08-10 21:42:39 +10:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/artifactapi#109
Delete Branch "benvin/github-auth"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?