feat: server-level GitHub machine credential for authenticated requests (#109)
ci/woodpecker/tag/docker Pipeline was successful

## Why

Anonymous GitHub is capped at 60 requests/hour and cannot read private repositories. A machine credential usable by a free (non-enterprise) account is needed to lift the request budget to ~5000/hr and to read private-repo release assets.

Builds on the background syncer (#108, now merged to `master`); this diff is the auth changes only.

## How

- Add `internal/githubauth`: a process-wide GitHub credential delivered via env/secret, applied by default to every outbound GitHub request (releases scan, ranged asset-header GETs, and the generic-github byte proxy for private assets).
- Support two modes:
  - **PAT** — `GITHUB_TOKEN` sent as `Authorization: Bearer <token>`.
  - **GitHub App** — `GITHUB_APP_ID` + `GITHUB_APP_INSTALLATION_ID` + private key (`GITHUB_APP_PRIVATE_KEY` inline PEM or `GITHUB_APP_PRIVATE_KEY_PATH`). Mint a short-lived RS256 JWT with stdlib `crypto/rsa` (no new dependency), exchange it at `POST /app/installations/{id}/access_tokens` for a ~1h installation token, cache it, and single-flight a refresh a few minutes before expiry.
- Inject at the two GitHub call paths: the rpm github provider header builder (releases + ranged fetches) and the generic provider `AuthHeaders` (byte proxy, github.com hosts only; the pre-signed `objects.githubusercontent.com` redirect deliberately gets no Authorization).
- Honor precedence: a remote's own `username`/`password` overrides the server credential; no credential configured stays anonymous (current behavior).
- Fail closed at startup on partial App configuration (e.g. App id without a private key); a token-and-App conflict is also rejected.
- Never persist the credential to the DB, return it from an API, or log it (token-exchange failures never echo the response body).
- Read config via the existing `getenv` convention; document PAT vs App setup, the free-account fine-grained PAT scopes (Contents:read + Metadata:read), precedence, and the rate-limit implication.

## Rate limit

Authenticated requests share the syncer's single global limiter — no second limiter is added. A token raises the effective GitHub ceiling (~5000/hr vs ~60/hr), so the limiter defaults stay safe.

## Tests

`internal/githubauth` and `internal/provider/{rpm,generic}`:
- PAT attaches the correct `Authorization` header to releases + asset-header requests.
- App mints a valid RS256 JWT (verified against the app public key), exchanges it at a mocked endpoint, reuses the cached token without re-exchanging, refreshes near expiry, and single-flights concurrent callers.
- Per-remote credential overrides the server credential (rpm + generic).
- No credential → no `Authorization` header, requests still succeed anonymously.
- ETag/304 flow still works with auth attached.
- The credential does not appear in a remote's serialized JSON.
- Config validation: no-config is anonymous; partial App config and token/App conflict both error.

Verified fail-before/pass-after for the injection tests. `gofmt -l`, `go build ./...`, `go vet ./...`, `go test ./...` all clean (26 packages).

Reviewed-on: #109
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
This commit was merged in pull request #109.
This commit is contained in:
2026-08-10 21:42:39 +10:00
committed by BenVincent
parent e24c35f534
commit 109ba2ce27
12 changed files with 968 additions and 12 deletions
+70
View File
@@ -111,6 +111,71 @@ resource "artifactapi_remote_github_rpm" "acme-tools" {
The repo is multi-arch (no `$basearch` needed) — `dnf` selects matching packages
from the synthesized metadata.
### GitHub authentication
Anonymous GitHub is capped at **60 requests/hour** and cannot read private
repositories. Configure a **server-level GitHub credential** to raise the ceiling
to roughly **5000 requests/hour** and to read private-repo release assets. The
credential is a process-wide machine identity applied by default to *every*
outbound GitHub request — the releases scan, the ranged asset-header fetches, and
the generic-github byte proxy that streams private release assets.
The credential is read from the environment (deliver it from a Vault or
Kubernetes secret). It is **never** stored per-remote in the database, **never**
returned by any API, and **never** logged. Configure **exactly one** mode.
**Precedence.** A remote's own `username`/`password` credential still wins for
that remote's requests; the server credential is the default for everything else.
With no credential configured at all, requests stay anonymous (current behavior).
Partial configuration (e.g. an App id with no private key) is a **startup error**
— artifactapi fails closed rather than silently falling back to anonymous.
Both modes share the syncer's single global rate limiter, so a token simply
raises the effective GitHub ceiling; the default limiter settings stay safe.
#### Mode 1 — Personal Access Token (minimum viable, recommended for free accounts)
Set `GITHUB_TOKEN`. It is sent as `Authorization: Bearer <token>`.
Recommended free-account setup — a **fine-grained PAT** scoped to just the target
repositories:
1. GitHub → *Settings → Developer settings → Personal access tokens →
Fine-grained tokens → Generate new token*.
2. Limit *Repository access* to the specific repo(s) serving releases.
3. Grant repository permissions **Contents: Read-only** and **Metadata:
Read-only** (Metadata is mandatory and auto-selected).
A classic PAT with the `repo` scope also works but is broader than necessary.
```bash
GITHUB_TOKEN=github_pat_xxxxxxxx
```
#### Mode 2 — GitHub App installation token (proper machine identity)
A GitHub App is not tied to a personal account and can be created and installed on
free personal repos. artifactapi mints a short-lived RS256 **JWT** from the app
private key, exchanges it at `POST /app/installations/{id}/access_tokens` for a
~1-hour **installation access token**, caches that token, and refreshes it a few
minutes before expiry (thread-safe, single-flighted).
1. GitHub → *Settings → Developer settings → GitHub Apps → New GitHub App*.
2. Under *Permissions → Repository permissions* grant **Contents: Read-only**
(Metadata: Read-only is implied).
3. Generate a **private key** (downloads a PEM) and note the **App ID**.
4. *Install* the App on the account and select the target repositories, then read
the **Installation ID** from the installation URL
(`.../settings/installations/<installation-id>`).
```bash
GITHUB_APP_ID=123456
GITHUB_APP_INSTALLATION_ID=7654321
GITHUB_APP_PRIVATE_KEY_PATH=/etc/artifactapi/github-app.pem
# or inline PEM (e.g. mounted from a secret):
# GITHUB_APP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----"
```
## Terraform
Remotes and virtuals are managed by Terraform. Each package type has its own resource:
@@ -277,6 +342,11 @@ S3 client supports MinIO, Ceph RGW, and AWS S3 (via minio-go).
| `GITHUB_SYNC_BURST` | `5` | Token-bucket burst for the shared limiter |
| `GITHUB_SYNC_WORKERS` | `3` | Concurrent `github_rpm` scan workers |
| `GITHUB_SYNC_POLL_INTERVAL` | `60` | Base scheduler tick in seconds; per-remote cadence is its `mutable_ttl`, enforced by the DB lease |
| `GITHUB_TOKEN` | | Server-level GitHub PAT (fine-grained or classic), sent as `Authorization: Bearer`. Applies to every GitHub request; per-remote creds override it. See [GitHub authentication](#github-authentication) |
| `GITHUB_APP_ID` | | GitHub App id (App auth mode; mutually exclusive with `GITHUB_TOKEN`) |
| `GITHUB_APP_INSTALLATION_ID` | | GitHub App installation id |
| `GITHUB_APP_PRIVATE_KEY` | | GitHub App private key, inline PEM |
| `GITHUB_APP_PRIVATE_KEY_PATH` | | GitHub App private key, file path (alternative to inline PEM) |
## Development