Files
artifactapi/internal/config/env.go
T
unkinben 109ba2ce27
ci/woodpecker/tag/docker Pipeline was successful
feat: server-level GitHub machine credential for authenticated requests (#109)
## Why

Anonymous GitHub is capped at 60 requests/hour and cannot read private repositories. A machine credential usable by a free (non-enterprise) account is needed to lift the request budget to ~5000/hr and to read private-repo release assets.

Builds on the background syncer (#108, now merged to `master`); this diff is the auth changes only.

## How

- Add `internal/githubauth`: a process-wide GitHub credential delivered via env/secret, applied by default to every outbound GitHub request (releases scan, ranged asset-header GETs, and the generic-github byte proxy for private assets).
- Support two modes:
  - **PAT** — `GITHUB_TOKEN` sent as `Authorization: Bearer <token>`.
  - **GitHub App** — `GITHUB_APP_ID` + `GITHUB_APP_INSTALLATION_ID` + private key (`GITHUB_APP_PRIVATE_KEY` inline PEM or `GITHUB_APP_PRIVATE_KEY_PATH`). Mint a short-lived RS256 JWT with stdlib `crypto/rsa` (no new dependency), exchange it at `POST /app/installations/{id}/access_tokens` for a ~1h installation token, cache it, and single-flight a refresh a few minutes before expiry.
- Inject at the two GitHub call paths: the rpm github provider header builder (releases + ranged fetches) and the generic provider `AuthHeaders` (byte proxy, github.com hosts only; the pre-signed `objects.githubusercontent.com` redirect deliberately gets no Authorization).
- Honor precedence: a remote's own `username`/`password` overrides the server credential; no credential configured stays anonymous (current behavior).
- Fail closed at startup on partial App configuration (e.g. App id without a private key); a token-and-App conflict is also rejected.
- Never persist the credential to the DB, return it from an API, or log it (token-exchange failures never echo the response body).
- Read config via the existing `getenv` convention; document PAT vs App setup, the free-account fine-grained PAT scopes (Contents:read + Metadata:read), precedence, and the rate-limit implication.

## Rate limit

Authenticated requests share the syncer's single global limiter — no second limiter is added. A token raises the effective GitHub ceiling (~5000/hr vs ~60/hr), so the limiter defaults stay safe.

## Tests

`internal/githubauth` and `internal/provider/{rpm,generic}`:
- PAT attaches the correct `Authorization` header to releases + asset-header requests.
- App mints a valid RS256 JWT (verified against the app public key), exchanges it at a mocked endpoint, reuses the cached token without re-exchanging, refreshes near expiry, and single-flights concurrent callers.
- Per-remote credential overrides the server credential (rpm + generic).
- No credential → no `Authorization` header, requests still succeed anonymously.
- ETag/304 flow still works with auth attached.
- The credential does not appear in a remote's serialized JSON.
- Config validation: no-config is anonymous; partial App config and token/App conflict both error.

Verified fail-before/pass-after for the injection tests. `gofmt -l`, `go build ./...`, `go vet ./...`, `go test ./...` all clean (26 packages).

Reviewed-on: #109
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-10 21:42:39 +10:00

137 lines
4.2 KiB
Go

package config
import (
"fmt"
"os"
"strconv"
)
type Config struct {
ListenAddr string
DBHost string
DBPort int
DBUser string
DBPass string
DBName string
DBSSL string
RedisURL string
S3Endpoint string
S3AccessKey string
S3SecretKey string
S3Bucket string
S3Secure bool
S3Region string
// Terraform provider registry signing. When TFSigningKeyPath points at a
// readable armored GPG private key, artifactapi serves local terraform
// repos as a real provider registry (service discovery + signed
// SHA256SUMS). Left empty, the registry endpoints stay disabled.
TFSigningKeyPath string
TFSigningKeyPassphrase string
TFProviderProtocols string
// github_rpm background syncer. The syncer keeps derived RPM metadata for
// every github_rpm remote fresh off the client request path, sharing a
// single global token-bucket limiter across all remotes so GitHub is never
// hammered. Defaults are conservative: 1 req/s (3600/hr) sits well under an
// authenticated token's 5000/hr. Unauthenticated remotes (60/hr) lean on
// ETag/304 — an unchanged repo costs nothing — so keep those repos small or
// configure a token.
GitHubSyncRatePerSec float64
GitHubSyncBurst int
GitHubSyncWorkers int
GitHubSyncPollInterval int
// Server-level GitHub machine credential, applied by default to every
// outbound GitHub request (releases scan, ranged asset fetches, and the
// generic-github byte proxy for private assets). Delivered via env/secret
// only — never stored per-remote, never returned by an API, never logged.
// Configure exactly one mode: a Personal Access Token, or a GitHub App
// (id + installation id + private key). Partial App config fails at startup.
GitHubToken string
GitHubAppID string
GitHubAppInstallationID string
GitHubAppPrivateKey string
GitHubAppPrivateKeyPath string
}
func (c *Config) DatabaseDSN() string {
return fmt.Sprintf(
"postgres://%s:%s@%s:%d/%s?sslmode=%s",
c.DBUser, c.DBPass, c.DBHost, c.DBPort, c.DBName, c.DBSSL,
)
}
func Load() (*Config, error) {
dbPort, err := strconv.Atoi(getenv("DBPORT", "5432"))
if err != nil {
return nil, fmt.Errorf("invalid DBPORT: %w", err)
}
s3Secure, _ := strconv.ParseBool(getenv("MINIO_SECURE", "false"))
syncRate, err := strconv.ParseFloat(getenv("GITHUB_SYNC_RATE", "1"), 64)
if err != nil {
return nil, fmt.Errorf("invalid GITHUB_SYNC_RATE: %w", err)
}
syncBurst, err := strconv.Atoi(getenv("GITHUB_SYNC_BURST", "5"))
if err != nil {
return nil, fmt.Errorf("invalid GITHUB_SYNC_BURST: %w", err)
}
syncWorkers, err := strconv.Atoi(getenv("GITHUB_SYNC_WORKERS", "3"))
if err != nil {
return nil, fmt.Errorf("invalid GITHUB_SYNC_WORKERS: %w", err)
}
syncPoll, err := strconv.Atoi(getenv("GITHUB_SYNC_POLL_INTERVAL", "60"))
if err != nil {
return nil, fmt.Errorf("invalid GITHUB_SYNC_POLL_INTERVAL: %w", err)
}
cfg := &Config{
ListenAddr: getenv("LISTEN_ADDR", ":8000"),
DBHost: getenv("DBHOST", "localhost"),
DBPort: dbPort,
DBUser: getenv("DBUSER", "artifacts"),
DBPass: getenv("DBPASS", ""),
DBName: getenv("DBNAME", "artifacts"),
DBSSL: getenv("DBSSL", "disable"),
RedisURL: getenv("REDIS_URL", "redis://localhost:6379"),
S3Endpoint: getenv("MINIO_ENDPOINT", "localhost:9000"),
S3AccessKey: getenv("MINIO_ACCESS_KEY", ""),
S3SecretKey: getenv("MINIO_SECRET_KEY", ""),
S3Bucket: getenv("MINIO_BUCKET", "artifacts"),
S3Secure: s3Secure,
S3Region: getenv("MINIO_REGION", ""),
TFSigningKeyPath: getenv("TF_SIGNING_KEY_PATH", ""),
TFSigningKeyPassphrase: getenv("TF_SIGNING_KEY_PASSPHRASE", ""),
TFProviderProtocols: getenv("TF_PROVIDER_PROTOCOLS", "5.0,6.0"),
GitHubSyncRatePerSec: syncRate,
GitHubSyncBurst: syncBurst,
GitHubSyncWorkers: syncWorkers,
GitHubSyncPollInterval: syncPoll,
GitHubToken: getenv("GITHUB_TOKEN", ""),
GitHubAppID: getenv("GITHUB_APP_ID", ""),
GitHubAppInstallationID: getenv("GITHUB_APP_INSTALLATION_ID", ""),
GitHubAppPrivateKey: getenv("GITHUB_APP_PRIVATE_KEY", ""),
GitHubAppPrivateKeyPath: getenv("GITHUB_APP_PRIVATE_KEY_PATH", ""),
}
return cfg, nil
}
func getenv(key, fallback string) string {
if v, ok := os.LookupEnv(key); ok {
return v
}
return fallback
}