## Why
Local terraform repos already served the Terraform **network mirror** protocol, but consuming that requires every user to add a `provider_installation { network_mirror }` block to `~/.terraformrc`. A `source = "artifactapi.k8s.../ns/type"` address instead triggers the **provider registry** protocol (service discovery at `/.well-known/terraform.json` + GPG-signed SHA256SUMS), which returned 404 — hence *"does not offer a provider registry."*
Local repos are meant to be the real thing, so this makes a terraform local repo a first-class provider registry: `terraform init` installs from a bare source address with no client config.
## What
- Serve `/.well-known/terraform.json` service discovery and the `providers.v1` endpoints under `/terraform/v1/providers`: `versions`, `download/{os}/{arch}`, `sha256sums`, `sha256sums.sig`.
- Map the Terraform **namespace** segment to the artifactapi **repo name**; locate the provider by **type**. `download_url` points back at the existing `/api/v1/local/...` path.
- Generate `SHA256SUMS` per version and sign it with a GPG key loaded from `TF_SIGNING_KEY_PATH` (optional `TF_SIGNING_KEY_PASSPHRASE`); advertise the public key + key id in the download response. **No key → registry stays disabled (endpoints 404)**, so behaviour is unchanged until the signing secret is present.
- New `internal/tfsign` (key load + detached signing, via `x/crypto/openpgp`) and `internal/api/terraform` (registry handler). Export `ParseProviderZip` for reuse.
- `TF_PROVIDER_PROTOCOLS` (default `5.0,6.0`) sets the advertised plugin protocols.
- README section documenting usage.
## Consumer
```hcl
terraform {
required_providers {
artifactapi = {
source = "artifactapi.k8s.syd1.au.unkin.net/terraform-unkin/artifactapi"
version = "0.1.2"
}
}
}
```
## Tests
- `internal/tfsign`: sign + verify round-trip, disabled/missing-key paths.
- `internal/api/terraform`: dockerised full flow (discovery → versions → download → sha256sums → sig), verifying the signature against the advertised public key.
## Follow-ups (separate PRs)
- **argocd-apps**: mount the signing K8s secret into the api deployment + set `TF_SIGNING_KEY_PATH`. The `/` HTTPRoute already routes `/.well-known` and `/terraform` to the API, so no gateway change is needed.
- Image/version bump once tagged.
## Note
Anchored the `terraform/` gitignore to the repo root (`/terraform/`) so it stops matching `internal/*/terraform/`. This surfaced `internal/provider/terraform/terraform_extra_test.go`, which had been silently untracked — now committed.
Reviewed-on: #102
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
ArtifactAPI
Caching proxy for package repositories. Single Go binary, 10 package types, content-addressable storage, managed by Terraform.
Quick Start
# Start backing services
docker compose up -d postgres redis minio
# Build and run
make build
./bin/artifactapi
# Frontend (separate container or dev server)
cd ui && npm install && npm run dev
API: http://localhost:8000 | Frontend: http://localhost:5173
Package Types
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|---|---|---|
generic |
nothing | everything |
docker |
tag manifests, /tags/list |
blobs, digest manifests |
helm |
index.yaml |
.tgz charts |
pypi |
simple/* index pages |
.whl, .tar.gz |
npm |
package metadata | .tgz tarballs |
rpm |
repomd.xml, repodata/* |
.rpm |
alpine |
APKINDEX.tar.gz |
.apk |
puppet |
v3/modules/*, v3/releases* |
.tar.gz |
terraform |
*/versions |
*/download/*/* |
goproxy |
@v/list, @latest |
.info, .mod, .zip |
Providers classify paths automatically. Users only configure what to proxy and TTLs.
Terraform
Remotes and virtuals are managed by Terraform. Each package type has its own resource:
resource "artifactapi_remote_generic" "github" {
name = "github"
base_url = "https://github.com"
immutable_ttl = 0
mutable_ttl = 7200
patterns = [
"ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$",
"mikefarah/yq/.*/yq_linux_amd64$",
]
mutable_patterns = [
".*/archive/refs/heads/.*\\.tar\\.gz$",
]
}
resource "artifactapi_remote_docker" "dockerhub" {
name = "dockerhub"
base_url = "https://registry-1.docker.io"
immutable_ttl = 0
mutable_ttl = 300
ban_tags_enabled = true
ban_tags = ["latest"]
patterns = [
"^library/postgres",
"^library/redis",
]
}
resource "artifactapi_remote_helm" "jetstack" {
name = "jetstack"
base_url = "https://charts.jetstack.io"
immutable_ttl = 0
mutable_ttl = 3600
}
resource "artifactapi_virtual" "helm" {
name = "helm"
package_type = "helm"
members = [artifactapi_remote_helm.jetstack.name]
}
Provider: terraform-provider-artifactapi
Serving providers as a registry
A local terraform repo is a real provider registry: upload
terraform-provider-{type}_{version}_{os}_{arch}.zip files under
{namespace}/{type}/, and Terraform installs them from a bare source address —
no .terraformrc mirror config:
terraform {
required_providers {
artifactapi = {
source = "artifactapi.k8s.syd1.au.unkin.net/<repo>/<type>"
version = "0.1.2"
}
}
}
The Terraform namespace segment is the artifactapi repo name; the provider is
matched by type. The registry serves service discovery
(/.well-known/terraform.json), the providers.v1 version/download endpoints,
and a GPG-signed SHA256SUMS per the provider registry protocol.
Signing needs a GPG key. By default artifactapi generates one on first start and
stores it in the database (signing_keys table), so every replica shares it and
there's nothing to provision. To bring your own key instead, point
TF_SIGNING_KEY_PATH at an armored private key (optionally
TF_SIGNING_KEY_PASSPHRASE), which takes precedence over the generated one.
TF_PROVIDER_PROTOCOLS (default 5.0,6.0) sets the advertised plugin protocols.
Access Control
| Field | Default | Behaviour |
|---|---|---|
patterns |
empty (proxy all) | If set, only matching paths are proxied. Acts as allowlist. |
blocklist |
empty | Matching paths always denied. Checked first. |
mutable_patterns |
empty | Override: force paths to mutable TTL. |
immutable_patterns |
empty | Override: force paths to immutable TTL. |
No patterns + no blocklist = open proxy. Provider handles mutability classification automatically.
API
Proxy (v1)
GET /api/v1/remote/{name}/{path} Proxy/cache artifact
GET /api/v1/virtual/{name}/{path} Virtual repo (merged index)
GET /v2/{name}/{path} Docker Registry v2
Management (v2)
GET/POST /api/v2/remotes List / create remotes
GET/PUT/DELETE /api/v2/remotes/{name} Read / update / delete remote
GET/DELETE /api/v2/remotes/{name}/objects Browse / evict cached objects
GET /api/v2/stats Overview stats
GET /api/v2/health Service health
POST /api/v2/probe Test a remote (fetch without streaming to client)
GET /api/v2/events SSE event stream
Architecture
PostgreSQL ─── config (remotes, virtuals), artifact metadata, access log
Redis ─── TTL keys, fetch locks, circuit breaker state
S3/MinIO ─── content-addressable blob storage (blobs/sha256/{hash})
S3 client supports MinIO, Ceph RGW, and AWS S3 (via minio-go).
Environment Variables
| Variable | Default | Description |
|---|---|---|
LISTEN_ADDR |
:8000 |
Server listen address |
DBHOST |
localhost |
PostgreSQL host |
DBPORT |
5432 |
PostgreSQL port |
DBUSER |
artifacts |
PostgreSQL user |
DBPASS |
PostgreSQL password | |
DBNAME |
artifacts |
PostgreSQL database |
REDIS_URL |
redis://localhost:6379 |
Redis URL |
MINIO_ENDPOINT |
localhost:9000 |
S3 endpoint |
MINIO_ACCESS_KEY |
S3 access key | |
MINIO_SECRET_KEY |
S3 secret key | |
MINIO_BUCKET |
artifacts |
S3 bucket |
MINIO_SECURE |
false |
Use HTTPS for S3 |
MINIO_REGION |
S3 region (AWS) |
Development
make build # Build binary
make test # Unit tests
make e2e # E2E tests (needs Docker)
make lint # golangci-lint + go vet
make fmt # gofmt + goimports
TUI
./bin/artifactapi tui --endpoint http://localhost:8000