unkin/artifactapi
2
0
Fork 0
Code Issues 5 Pull Requests 2 Actions Packages Projects Releases 1 Wiki Activity
Files
9287cf7cf2a77b855e4922942dc661b2d8a2e868
artifactapi/examples/single-file/remotes.yaml
T
unkinben 9287cf7cf2 feat: add Puppet Forge remote type (#44) 
## Summary

- Adds \`package: puppet\` for proxying Puppet Forge (forgeapi.puppet.com)
- \`remote/puppet.py\` rewrites JSON responses: absolute forge URLs → proxy URLs, and relative \`/v3/files/\` \`file_uri\` paths → absolute proxy URLs. g10k uses Go's \`url.ResolveReference\`, so an absolute \`file_uri\` overrides the base URL entirely — tarballs are fetched directly from the proxy without a second hop
- Built-in mutable patterns: \`^v3/modules/\` and \`^v3/releases\` (module metadata); tarballs at \`v3/files/\` are configured as immutable via \`immutable_patterns\`
- 9 new tests covering mutable detection, URL rewriting (relative \`file_uri\` and absolute forge URLs), content-type, tarball pass-through, and pattern blocking

## Client configuration

**g10k config file** (\`forge_base_url\` at root level):
\`\`\`yaml
cachedir: /tmp/g10k
forge_base_url: https://artifacts.example.com/api/v1/remote/puppet-forge
sources:
  control:
    remote: git@git.example.com:puppet/control.git
    basedir: /etc/puppetlabs/code/environments
\`\`\`

**Puppetfile** (\`forge.baseUrl\` directive, works with \`-puppetfile\` mode):
\`\`\`ruby
forge.baseUrl https://artifacts.example.com/api/v1/remote/puppet-forge

mod 'puppetlabs-stdlib', '9.7.0'
\`\`\`

## Test plan

- [x] 331 unit tests pass (\`make test\`)
- [x] End-to-end: g10k 0.9.10 on AlmaLinux 9 via \`forge_base_url\` — stdlib 9.7.0, inifile 6.2.0, concat 9.1.0 installed; proxy logs confirm cache MISS → fetch → ADD for metadata and tarballs
- [x] End-to-end: \`forge.baseUrl\` Puppetfile directive with \`-puppetfile\` mode — same result

Reviewed-on: #44
2026-05-17 10:56:50 +10:00

502 lines
16 KiB
YAML
Raw Blame History

# Example remotes configuration — copy and adapt for your environment.
#
# immutable_patterns: artifacts cached forever (e.g. release binaries, versioned tags).
# mutable_patterns: artifacts that expire after cache.mutable_ttl seconds and are
# re-fetched from upstream on next request (e.g. index files,
# branch archives). Defaults to the package-type built-ins when
# not set (APKINDEX, repomd.xml, Docker manifests, etc.).
# cache:
# immutable_ttl: TTL for immutable files (0 = forever, rarely needed to change).
# mutable_ttl: TTL in seconds for mutable files. Omit to use the default (3600).
#
# quarantine_new: Set to true to block immutable artifacts published within the last
# quarantine_days days. Requests return 404 until the quarantine period
# expires. Fails open when the publish date cannot be determined.
# quarantine_days: Number of days to quarantine newly published artifacts (requires
# quarantine_new: true). The upstream Last-Modified header is used as
# the publish date.
#
# WARNING: this file may contain credentials — do not commit real values.
#
# Global configuration
#s3:
# endpoint: "localhost:9000"
# access_key: "minioadmin"
# secret_key: "minioadmin"
# bucket: "artifacts"
# secure: false
#
#redis:
# url: "redis://localhost:6379/0"
#
#database:
# url: "postgresql://artifacts:artifacts123@localhost:5432/artifacts"
#
remotes:
github:
base_url: "https://github.com"
package: "generic"
description: "GitHub releases and files"
immutable_patterns:
- "gruntwork-io/terragrunt/.*terragrunt_linux_amd64.*"
- "lxc/incus/.*\\.tar\\.gz$"
- "prometheus/node_exporter/.*/node_exporter-.*\\.linux-amd64\\.tar\\.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vmutils-linux-amd64-.*\\.tar\\.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-metrics-linux-amd64-.*-cluster\\.tar\\.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/victoria-logs-linux-amd64-.*\\.tar\\.gz$"
- "VictoriaMetrics/VictoriaMetrics/.*/vlutils-linux-amd64-.*\\.tar\\.gz$"
- "prometheus-community/bind_exporter/.*/bind_exporter-.*\\.linux-amd64\\.tar\\.gz$"
- "prometheus-community/pgbouncer_exporter/.*/pgbouncer_exporter-.*\\.linux-amd64\\.tar\\.gz$"
- "prometheus-community/postgres_exporter/.*/postgres_exporter-.*\\.linux-amd64\\.tar\\.gz$"
- "onedr0p/exportarr/.*/exportarr_.*_linux_amd64\\.tar\\.gz$"
- "tynany/frr_exporter/.*/frr_exporter-.*\\.linux-amd64\\.tar\\.gz$"
- "camptocamp/prometheus-puppetdb-exporter/.*/prometheus-puppetdb-exporter-.*\\.linux-amd64\\.tar\\.gz$"
- "grafana/jsonnet-language-server/.*/jsonnet-language-server_.*_linux_amd64$"
- "helmfile/helmfile/.*/helmfile_.*_linux_amd64\\.tar\\.gz$"
- "helmfile/vals/.*/vals_.*_linux_amd64\\.tar\\.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-consul_linux_amd64_.*\\.tar\\.gz$"
- "openbao/openbao-plugins/.*/openbao-plugin-secrets-nomad_linux_amd64_.*\\.tar\\.gz$"
- "apple/foundationdb/.*/libfdb_c\\.x86_64\\.so$"
- "stalwartlabs/stalwart/.*/stalwart-cli-x86_64-unknown-linux-gnu\\.tar\\.gz$"
- "stalwartlabs/stalwart/.*/stalwart-foundationdb-x86_64-unknown-linux-gnu\\.tar\\.gz$"
- "stalwartlabs/stalwart/.*/stalwart-x86_64-unknown-linux-gnu\\.tar\\.gz$"
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 0
github-archive:
base_url: "https://github.com"
package: "generic"
description: "GitHub repository archive tarballs"
immutable_patterns:
# Tag archives are immutable — a tag never changes
- ".*/archive/refs/tags/.*\\.tar\\.gz$"
mutable_patterns:
# Branch archives can change on every push
- ".*/archive/refs/heads/main\\.tar\\.gz$"
- ".*/archive/refs/heads/master\\.tar\\.gz$"
# Before re-downloading an expired branch archive, check whether it has
# actually changed (304 Not Modified → just refresh the TTL, no transfer).
# Only applies to user-defined mutable_patterns, not package-type defaults.
check_mutable_updates: true
cache:
immutable_ttl: 0 # Tag archives cached indefinitely
mutable_ttl: 86400 # Branch archives refreshed after 1 day
gitea-dl:
base_url: "https://dl.gitea.com"
package: "generic"
description: "Gitea download site"
immutable_patterns:
- "act_runner/.*/act_runner-.*-linux-amd64$"
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 0
hashicorp-releases:
base_url: "https://releases.hashicorp.com"
package: "generic"
description: "HashiCorp product releases"
immutable_patterns:
- "terraform/.*terraform_.*_linux_amd64\\.zip$"
- "terraform/.*terraform_.*_windows_amd64\\.zip$"
- "terraform/.*terraform_.*_darwin_amd64\\.zip$"
- "vault/.*vault_.*_linux_amd64\\.zip$"
- "vault/.*vault_.*_windows_amd64\\.zip$"
- "vault/.*vault_.*_darwin_amd64\\.zip$"
- "consul-cni/.*/consul-cni_.*_linux_amd64\\.zip$"
- "consul/.*/consul_.*_linux_amd64\\.zip$"
- "nomad-autoscaler/.*/nomad-autoscaler_.*_linux_amd64\\.zip$"
- "nomad/.*/nomad_.*_linux_amd64\\.zip$"
- "packer/.*/packer_.*_linux_amd64\\.zip$"
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 0
alpine:
base_url: "https://dl-cdn.alpinelinux.org"
package: "alpine"
description: "Alpine Linux APK package repository"
immutable_patterns:
- ".*/x86_64/.*\\.apk$"
# check_mutable_updates not set: APKINDEX.tar.gz is a package-type default
# and is always re-fetched on expiry — conditional checks are skipped for
# built-in mutable patterns regardless of this flag.
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 7200 # Index files (APKINDEX.tar.gz) cached for 2 hours
almalinux:
base_url: "https://gsl-syd.mm.fcix.net/almalinux"
package: "rpm"
description: "AlmaLinux RPM package repository"
immutable_patterns:
- ".*/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*$"
- ".*\\.rpm$" # Allow all RPM files
# repomd.xml / repodata are package-type defaults — always re-fetched on
# expiry. check_mutable_updates would only apply to any custom
# mutable_patterns added here.
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 7200 # Metadata files cached for 2 hours
epel:
base_url: "http://mirror.aarnet.edu.au/pub/epel"
package: "rpm"
description: "EPEL (Extra Packages for Enterprise Linux)"
immutable_patterns:
- "8/Everything/x86_64/.*\\.rpm$"
- "9/Everything/x86_64/.*\\.rpm$"
- "10/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- ".*/repodata/.*$"
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 7200 # Metadata files cached for 2 hours
fedora:
base_url: "https://gsl-syd.mm.fcix.net/fedora/linux"
package: "rpm"
description: "Fedora Linux RPM package repository"
immutable_patterns:
- "releases/.*/Everything/x86_64/.*\\.rpm$"
- "updates/.*/Everything/x86_64/.*\\.rpm$"
- "development/.*/Everything/x86_64/.*\\.rpm$"
- ".*/noarch/.*\\.rpm$"
- "updates/.*/Everything/x86_64/repodata/.*$"
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 300 # Metadata files cached for 5 minutes
ghcr:
base_url: "https://ghcr.io"
package: "docker"
description: "GitHub Container Registry"
# username: "your-github-username"
# password: "your-github-pat" # needs read:packages scope
# Docker manifest/tag-list patterns are package-type defaults — always
# re-fetched on expiry. check_mutable_updates only applies to any custom
# mutable_patterns you add (e.g. a metadata endpoint).
cache:
immutable_ttl: 0
mutable_ttl: 300
dockerhub:
base_url: "https://registry-1.docker.io"
package: "docker"
description: "Docker Hub registry"
cache:
immutable_ttl: 0
mutable_ttl: 300
pypi:
base_url: "https://files.pythonhosted.org"
package: "pypi"
description: "Python Package Index — simple index and package files via a single remote"
# simple/ requests are transparently fetched from pypi.org; package files come from
# files.pythonhosted.org (base_url). URLs in the simple index are rewritten to this remote.
check_mutable_updates: true
# Block packages published within the last 3 days (supply-chain attack mitigation).
# Immutable artifacts (wheel/sdist) newer than quarantine_days return 404 until
# the window passes. Disable by setting quarantine_new: false or removing both keys.
quarantine_new: true
quarantine_days: 3
immutable_patterns:
- "packages/.*\\.whl$"
- "packages/.*\\.whl\\.metadata$"
- "packages/.*\\.tar\\.gz$"
- "packages/.*\\.zip$"
- "packages/.*\\.egg$"
cache:
immutable_ttl: 0
mutable_ttl: 600 # Simple index pages refreshed after 10 minutes
pypi-gitea:
base_url: "https://gitea.example.com/api/packages/myorg/pypi"
package: "pypi"
description: "Private Gitea PyPI registry — simple index and files at the same host"
# username: "your-gitea-username"
# password: "your-personal-access-token" # needs package:read scope
check_mutable_updates: true
immutable_patterns:
- "files/.*\\.whl$"
- "files/.*\\.whl\\.metadata$"
- "files/.*\\.tar\\.gz$"
- "files/.*\\.zip$"
- "files/.*\\.egg$"
cache:
immutable_ttl: 0
mutable_ttl: 600
npm:
base_url: "https://registry.npmjs.org"
package: "npm"
description: "npm registry — package metadata with tarball URL rewriting"
check_mutable_updates: true
immutable_patterns:
- \.tgz$
mutable_patterns:
- ^(?!.*\.tgz$).*
cache:
immutable_ttl: 0
mutable_ttl: 600 # Package metadata refreshed after 10 minutes
hashicorp-helm:
base_url: "https://helm.releases.hashicorp.com"
package: "helm"
description: "HashiCorp Helm chart repository (Vault, Consul, Nomad, etc.)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0 # Chart tarballs are versioned — cache forever
mutable_ttl: 3600 # index.yaml refreshed after 1 hour
metallb:
base_url: "https://metallb.github.io/metallb"
package: "helm"
description: "MetalLB load balancer Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
jetstack:
base_url: "https://charts.jetstack.io"
package: "helm"
description: "Jetstack Helm charts (cert-manager)"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
rancher-stable:
base_url: "https://releases.rancher.com/server-charts/stable"
package: "helm"
description: "Rancher stable Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
purelb:
base_url: "https://gitlab.com/api/v4/projects/20400619/packages/helm/stable"
package: "helm"
description: "PureLB load balancer Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
istio:
base_url: "https://istio-release.storage.googleapis.com/charts"
package: "helm"
description: "Istio service mesh Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
cnpg:
base_url: "https://cloudnative-pg.github.io/charts"
package: "helm"
description: "CloudNativePG operator Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
ceph-csi:
base_url: "https://ceph.github.io/csi-charts"
package: "helm"
description: "Ceph CSI driver Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
external-dns:
base_url: "https://kubernetes-sigs.github.io/external-dns/"
package: "helm"
description: "ExternalDNS Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
intel-helm:
base_url: "https://intel.github.io/helm-charts/"
package: "helm"
description: "Intel Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
elastic:
base_url: "https://helm.elastic.co"
package: "helm"
description: "Elastic stack Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
k8up-io:
base_url: "https://k8up-io.github.io/k8up"
package: "helm"
description: "K8up backup operator Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
victoriametrics:
base_url: "https://victoriametrics.github.io/helm-charts/"
package: "helm"
description: "VictoriaMetrics observability Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
grafana:
base_url: "https://grafana.github.io/helm-charts"
package: "helm"
description: "Grafana observability Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
helm-openldap:
base_url: "https://jp-gouin.github.io/helm-openldap/"
package: "helm"
description: "OpenLDAP Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
woodpecker:
base_url: "https://woodpecker-ci.org/"
package: "helm"
description: "Woodpecker CI Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
stakater:
base_url: "https://stakater.github.io/stakater-charts"
package: "helm"
description: "Stakater Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
jfrog:
base_url: "https://charts.jfrog.io/"
package: "helm"
description: "JFrog Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
openvox:
base_url: "https://openvoxproject.github.io/openvox-helm-chart"
package: "helm"
description: "OpenVox Helm charts"
check_mutable_updates: true
immutable_patterns:
- "\\.tgz$"
cache:
immutable_ttl: 0
mutable_ttl: 3600
puppet-forge:
base_url: "https://forgeapi.puppet.com"
package: "puppet"
description: "Puppet Forge module registry"
# Module metadata (v3/modules/, v3/releases) is mutable by default.
# Configure r10k / librarian-puppet with this remote as the Forge URL:
# http://your-proxy/api/v1/remote/puppet-forge
check_mutable_updates: true
immutable_patterns:
- "^v3/files/.*\\.tar\\.gz$"
cache:
immutable_ttl: 0 # Module tarballs cached indefinitely
mutable_ttl: 600 # Module metadata refreshed after 10 minutes
virtuals:
helm-all:
package: "helm"
description: "Virtual repository merging all helm remotes — member order is priority order for duplicate chart+version"
members:
- hashicorp-helm
- metallb
- jetstack
- rancher-stable
- purelb
- istio
- cnpg
- ceph-csi
- external-dns
- intel-helm
- elastic
- k8up-io
- victoriametrics
- grafana
- helm-openldap
- woodpecker
- stakater
- jfrog
- openvox
locals:
local-generic:
package: "generic"
description: "Local generic file repository"
cache:
immutable_ttl: 0 # Files cached indefinitely
mutable_ttl: 0
Reference in New Issue View Git Blame Copy Permalink