unkinben edb6c7c0f7
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
feat: serve local terraform repos as a provider registry
Local terraform repos already spoke the network mirror protocol, which needs
per-consumer .terraformrc config. This adds the provider registry protocol so
`terraform init` installs from a bare source address
(artifactapi.k8s.../{repo}/{type}) with no client setup.

- serve /.well-known/terraform.json service discovery and the providers.v1
  versions/download endpoints under /terraform/v1/providers
- map the Terraform namespace to the artifactapi repo name and locate the
  provider by type; download_url points back at the existing local file path
- generate SHA256SUMS per version and sign it with a GPG key loaded from
  TF_SIGNING_KEY_PATH; advertise the public key + key id in the download
  response. No key configured -> registry stays disabled (endpoints 404)
- new internal/tfsign (key loading + detached signing) and
  internal/api/terraform (registry handler); export ParseProviderZip for reuse
- add TF_SIGNING_KEY_PATH/PASSPHRASE and TF_PROVIDER_PROTOCOLS config
- unit test signing + verification; dockerised test of the full flow incl.
  signature verification against the advertised key

Also anchor the terraform/ gitignore to the repo root so it stops swallowing
internal/api/terraform and internal/provider/terraform test files (the latter
had gone silently untracked).
2026-07-03 17:46:55 +10:00
2026-06-07 19:30:35 +10:00

ArtifactAPI

Caching proxy for package repositories. Single Go binary, 10 package types, content-addressable storage, managed by Terraform.

Quick Start

# Start backing services
docker compose up -d postgres redis minio

# Build and run
make build
./bin/artifactapi

# Frontend (separate container or dev server)
cd ui && npm install && npm run dev

API: http://localhost:8000 | Frontend: http://localhost:5173

Package Types

Type Mutable (auto-detected) Immutable (auto-detected)
generic nothing everything
docker tag manifests, /tags/list blobs, digest manifests
helm index.yaml .tgz charts
pypi simple/* index pages .whl, .tar.gz
npm package metadata .tgz tarballs
rpm repomd.xml, repodata/* .rpm
alpine APKINDEX.tar.gz .apk
puppet v3/modules/*, v3/releases* .tar.gz
terraform */versions */download/*/*
goproxy @v/list, @latest .info, .mod, .zip

Providers classify paths automatically. Users only configure what to proxy and TTLs.

Terraform

Remotes and virtuals are managed by Terraform. Each package type has its own resource:

resource "artifactapi_remote_generic" "github" {
  name     = "github"
  base_url = "https://github.com"

  immutable_ttl = 0
  mutable_ttl   = 7200

  patterns = [
    "ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$",
    "mikefarah/yq/.*/yq_linux_amd64$",
  ]

  mutable_patterns = [
    ".*/archive/refs/heads/.*\\.tar\\.gz$",
  ]
}

resource "artifactapi_remote_docker" "dockerhub" {
  name     = "dockerhub"
  base_url = "https://registry-1.docker.io"

  immutable_ttl    = 0
  mutable_ttl      = 300
  ban_tags_enabled = true
  ban_tags         = ["latest"]

  patterns = [
    "^library/postgres",
    "^library/redis",
  ]
}

resource "artifactapi_remote_helm" "jetstack" {
  name     = "jetstack"
  base_url = "https://charts.jetstack.io"

  immutable_ttl = 0
  mutable_ttl   = 3600
}

resource "artifactapi_virtual" "helm" {
  name         = "helm"
  package_type = "helm"
  members      = [artifactapi_remote_helm.jetstack.name]
}

Provider: terraform-provider-artifactapi

Serving providers as a registry

A local terraform repo is a real provider registry: upload terraform-provider-{type}_{version}_{os}_{arch}.zip files under {namespace}/{type}/, and Terraform installs them from a bare source address — no .terraformrc mirror config:

terraform {
  required_providers {
    artifactapi = {
      source  = "artifactapi.k8s.syd1.au.unkin.net/<repo>/<type>"
      version = "0.1.2"
    }
  }
}

The Terraform namespace segment is the artifactapi repo name; the provider is matched by type. The registry serves service discovery (/.well-known/terraform.json), the providers.v1 version/download endpoints, and a GPG-signed SHA256SUMS per the provider registry protocol.

Signing requires an armored GPG private key, supplied via TF_SIGNING_KEY_PATH (optionally TF_SIGNING_KEY_PASSPHRASE). Without it the registry endpoints stay disabled. TF_PROVIDER_PROTOCOLS (default 5.0,6.0) sets the advertised plugin protocols.

Access Control

Field Default Behaviour
patterns empty (proxy all) If set, only matching paths are proxied. Acts as allowlist.
blocklist empty Matching paths always denied. Checked first.
mutable_patterns empty Override: force paths to mutable TTL.
immutable_patterns empty Override: force paths to immutable TTL.

No patterns + no blocklist = open proxy. Provider handles mutability classification automatically.

API

Proxy (v1)

GET /api/v1/remote/{name}/{path}     Proxy/cache artifact
GET /api/v1/virtual/{name}/{path}    Virtual repo (merged index)
GET /v2/{name}/{path}                Docker Registry v2

Management (v2)

GET/POST        /api/v2/remotes              List / create remotes
GET/PUT/DELETE  /api/v2/remotes/{name}       Read / update / delete remote
GET/DELETE      /api/v2/remotes/{name}/objects  Browse / evict cached objects
GET             /api/v2/stats                Overview stats
GET             /api/v2/health               Service health
POST            /api/v2/probe                Test a remote (fetch without streaming to client)
GET             /api/v2/events               SSE event stream

Architecture

PostgreSQL  ─── config (remotes, virtuals), artifact metadata, access log
Redis       ─── TTL keys, fetch locks, circuit breaker state
S3/MinIO    ─── content-addressable blob storage (blobs/sha256/{hash})

S3 client supports MinIO, Ceph RGW, and AWS S3 (via minio-go).

Environment Variables

Variable Default Description
LISTEN_ADDR :8000 Server listen address
DBHOST localhost PostgreSQL host
DBPORT 5432 PostgreSQL port
DBUSER artifacts PostgreSQL user
DBPASS PostgreSQL password
DBNAME artifacts PostgreSQL database
REDIS_URL redis://localhost:6379 Redis URL
MINIO_ENDPOINT localhost:9000 S3 endpoint
MINIO_ACCESS_KEY S3 access key
MINIO_SECRET_KEY S3 secret key
MINIO_BUCKET artifacts S3 bucket
MINIO_SECURE false Use HTTPS for S3
MINIO_REGION S3 region (AWS)

Development

make build       # Build binary
make test        # Unit tests
make e2e         # E2E tests (needs Docker)
make lint        # golangci-lint + go vet
make fmt         # gofmt + goimports

TUI

./bin/artifactapi tui --endpoint http://localhost:8000
S
Description
My terrible vibe coded artifact cache
Readme 1.9 MiB
Languages
Go 85.2%
TypeScript 10.9%
CSS 2.9%
Makefile 0.5%
Shell 0.3%