Commit Graph

51 Commits

Author SHA1 Message Date
unkin-agent 62d5e2fb14 ghcr remote: allow goodtune/ghp for pull-through
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-08-13 19:31:01 +10:00
benvin 2ce7478b5e Merge pull request 'fedora remote: switch base_url to syd.mirror.rackspace.com (fast AU mirror w/ F44)' (#23) from benvin/fedora-mirror-unistuttgart into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #23
2026-08-13 07:02:05 +10:00
unkin-agent 1f12b513ea fedora remote: switch base_url to syd.mirror.rackspace.com
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-08-13 06:42:27 +10:00
unkin-agent e0236b0584 fedora remote: switch base_url to ftp.uni-stuttgart.de
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
dl.fedoraproject.org is the throttled master, causing F44 cold-fetch
timeouts. uni-stuttgart is a fast tier-1 mirror carrying the full
releases+updates tree.
2026-08-13 00:47:19 +10:00
benvin 950a228fdc Merge pull request 'fedora remote: point base_url at dl.fedoraproject.org' (#22) from benvin/fedora-mirror-dl into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #22
2026-08-12 23:13:51 +10:00
unkin-agent 983cff4876 fedora remote: point base_url at dl.fedoraproject.org
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
The fcix micromirror (gsl-syd.mm.fcix.net) does not carry F44, so dnf
gets HTTP 404 for releases/44/Everything/x86_64/os/repodata/repomd.xml.
dl.fedoraproject.org is the canonical always-current Fedora source.
2026-08-12 22:42:45 +10:00
benvin 734f1026e2 Merge pull request 'Bump artifactapi provider to v0.2.0' (#21) from benvin/provider-v0.2.0 into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #21
2026-08-10 23:32:51 +10:00
benvin f1f6d699e5 Bump artifactapi provider to v0.2.0
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Pin the artifactapi Terraform provider to v0.2.0, which adds the artifactapi_remote_github_rpm resource (GitHub releases served as a yum repo). Verified v0.2.0 resolves and installs from the terraform-unkin registry.
2026-08-10 23:28:07 +10:00
benvin 0031eda691 Merge pull request 'Add kubernetes rpm remote (pkgs.k8s.io, any stable minor)' (#20) from benvin/kubernetes-rpm-remote into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #20
2026-07-30 21:24:50 +10:00
unkinben 83e4dbd00d fix: quote base_url (trailing colon broke yaml parse)
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-30 21:16:01 +10:00
unkinben df74a5d02a Add kubernetes rpm remote (pkgs.k8s.io, any stable minor)
ci/woodpecker/pr/plan Pipeline failed
ci/woodpecker/pr/pre-commit Pipeline failed
Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-30 21:10:59 +10:00
benvin 1da1a1ae40 Merge pull request 'Add local_generic rootfs-images repo (bootapi images)' (#18) from benvin/local-generic-rootfs into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #18
2026-07-30 00:14:06 +10:00
unkinben 8ecf5f11da ci: re-run plan with provider v0.1.4 released
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-29 23:56:23 +10:00
benvin 4386c7ea16 Merge pull request 'Allow the Gitea image through the dockerhub remote' (#19) from benvin/gitea-images into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #19
2026-07-29 23:56:06 +10:00
unkinben fd3bd37ba0 Allow the Gitea image through the dockerhub remote
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
The k8s Gitea deployment pulls gitea/gitea (rootless) from Docker Hub via the
artifactapi mirror. Add it to the dockerhub allowlist.

- add "^gitea/gitea" to config/remote_docker/dockerhub.yaml patterns

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-29 22:02:09 +10:00
unkinben 36568ca53e Add local_generic rootfs-images repository
ci/woodpecker/pr/plan Pipeline failed
ci/woodpecker/pr/pre-commit Pipeline failed
Declares a local generic (raw-file) repo for bootapi's node rootfs tarballs and
wires the local_generic kind through the module. Depends on
terraform-provider-artifactapi #12 releasing v0.1.4 (adds artifactapi_local_generic);
the provider version pin is bumped to 0.1.4, so plan/validate stay red until that
release is available.

- config/local_generic/rootfs-images.yaml (new repo).
- local_generic wiring: config.hcl, module variables.tf + main.tf, env
  terragrunt.hcl inputs; provider pin 0.1.3 -> 0.1.4.

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-29 21:51:10 +10:00
benvin 3a0b328a6f Merge pull request 'Allow the NetBox image through the ghcr remote' (#17) from benvin/netbox-ghcr-allowlist into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #17
2026-07-28 17:28:21 +10:00
unkinben 0d4fd7a3f5 Allow the NetBox image through the ghcr remote
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
NetBox is being deployed to k8s (argocd-apps) as the source of truth for
host/interface/IPAM data. Its image is ghcr.io/netbox-community/netbox, pulled
through the artifactapi ghcr mirror, so add the ^netbox-community/ pattern to
the ghcr remote allowlist.

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-28 16:44:52 +10:00
benvin 69218547d9 Merge pull request 'Allow logging-stack images through the dockerhub remote' (#16) from benvin/logging-stack-docker-remotes into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #16
2026-07-27 21:46:50 +10:00
unkinben fbc6f6129d Allow logging-stack images through the dockerhub remote
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
The centralized logging stack (argocd-apps ClickHouse + Vector + NATS
JetStream) pulls all container images through artifactapi instead of directly
from upstream. Add the Docker Hub repositories it needs to the dockerhub remote
allowlist.

- add ^altinity/ (clickhouse-operator + metrics-exporter)
- add ^clickhouse/ (clickhouse-server)
- add ^library/nats (NATS server)
- add ^natsio/ (nats-box, nats-server-config-reloader)
- add ^timberio/vector (vector agent/aggregator/archiver/vm-ingest + CI)

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-27 21:15:47 +10:00
benvin 67d9296e09 Merge pull request 'Add distribution-agnostic rpm-vendor local repo' (#15) from benvin/rpm-vendor-general into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #15
2026-07-26 23:14:15 +10:00
unkinben cf2b7ef19c Add distribution-agnostic rpm-vendor local repo
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
The puppet AlmaLinux hosts now consume a general rpm-vendor repo (alongside
the per-distro rpm-vendor-elN repos), mirroring the rpm-internal split. Add
the backing local rpm repo so the baseurl resolves.

- add config/local_rpm/rpm-vendor.yaml

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-26 23:08:48 +10:00
benvin fd22d192f3 Merge pull request 'Allow VPA CRD manifest through the github remote' (#14) from benvin/vpa-crd-pattern into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #14
2026-07-25 17:48:20 +10:00
unkinben e59e654fd2 Allow VPA CRD manifest through the github remote
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-07-25 17:32:57 +10:00
benvin a6819e2232 Merge pull request 'Add autoscaling/vpa-* pattern to k8s-registry docker remote' (#13) from benvin/vpa-remotes into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #13
2026-07-25 17:21:31 +10:00
unkinben 55d4757553 Add autoscaling/vpa-* pattern to k8s-registry docker remote
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
The VerticalPodAutoscaler rollout pulls registry.k8s.io/autoscaling/vpa-recommender.
Serve it through the existing k8s-registry pull-through cache, consistent with how
external-dns and sig-storage images from registry.k8s.io are already gated.
2026-07-25 17:04:45 +10:00
benvin bdbe26a6f3 Merge pull request 'Add fedora-archive remote for EOL releases (F42)' (#12) from benvin/fedora-archive into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #12
2026-07-25 16:43:51 +10:00
unkinben 449fff0ab3 Add fedora-archive remote for EOL releases (F42)
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
2026-07-25 16:20:56 +10:00
benvin 897d7b5c8f Merge pull request 'ci: fetch vault from artifactapi instead of dnf install' (#11) from benvin/ci-vault-install-speedup into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #11
2026-07-25 09:46:47 +10:00
unkinben 71f431abfd Escape VAULT_VERSION for woodpecker YAML substitution ($$ -> shell)
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-07-25 00:32:21 +10:00
benvin 014a79ddee ci: fetch vault from artifactapi instead of dnf install
ci/woodpecker/pr/plan Pipeline failed
ci/woodpecker/pr/pre-commit Pipeline was successful
CI installed vault by shelling out to `dnf install vault -y`, which reads
metadata for every enabled repo (appstream/baseos/crb/epel/ha) and downloads
the 169MB vendored vault RPM from the unkin repo on every pipeline run
(~39s per plan/apply job).

- Replace the dnf install with a pinned curl of the upstream vault zip from
  the artifactapi hashicorp-releases remote proxy, extracted with python3 to
  /usr/local/bin/vault.
- Pin the version via a VAULT_VERSION env var (1.20.0).
2026-07-25 00:27:21 +10:00
benvin 70caa416d7 Merge pull request 'Allowlist the iplocate DB patterns' (#10) from benvin/iplocate-patterns-allowlist into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #10
2026-07-24 00:15:35 +10:00
benvin dd48e7a8e0 Allowlist the iplocate DB patterns (add to patterns, not just mutable_patterns)
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
The iplocate patterns were added to mutable_patterns only, which tags TTL but
does not allowlist — requests 403'd 'access denied'. Verified: existing
mutable_patterns-only paths (branch archives, webadmin) also 403, while patterns
entries (uv) return 200. Add the iplocate patterns to patterns (the allowlist),
keeping them in mutable_patterns so the daily-updated DB stays mutable.
2026-07-24 00:13:36 +10:00
benvin 71b0918b54 Merge pull request 'Proxy iplocate IP databases via the github remote' (#9) from benvin/add-iplocate-remote into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #9
2026-07-22 00:52:20 +10:00
benvin 36e23e9a2b Proxy iplocate IP databases via the existing github remote
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
Add patterns to the github.com generic remote for the iplocate ip-to-asn /
ip-to-country databases rather than standing up a new remote. The files are
Git-LFS, so the /raw/ path redirects to media.githubusercontent.com; the github
remote already follows github's cross-host redirects (as it does for release
assets), and the raw path returns the real bytes (verified: 12MB zip / 78MB mmdb).
Marked mutable (branch content).
2026-07-21 22:33:01 +10:00
benvin 8735cc7e74 Merge pull request 'rpm: proxy RPM Fusion free/nonfree remotes' (#7) from benvin/rpmfusion-remotes into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #7
2026-07-18 14:29:45 +10:00
unkinben e71eb90101 rpm: drop repodata from rpmfusion immutable_patterns
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
repodata is regenerated upstream, so it must stay mutable (mutable_ttl)
rather than being cached as immutable.

Claude-Session: https://claude.ai/code/session_01VZ81u2JtjvaPq8Z3EDTSMb
2026-07-18 14:18:35 +10:00
unkinben 79f4d244e8 Merge pull request 'remote_rpm: stop caching repodata as immutable' (#8) from benvin/rpm-repodata-mutable into main
ci/woodpecker/push/apply Pipeline was successful
2026-07-18 14:15:52 +10:00
unkinben d168962f82 remote_rpm: stop caching repodata as immutable
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
repodata (repomd.xml and its referenced index files) is a mutable index that
upstream RPM repos rotate frequently. Marking it immutable (immutable_ttl: 0)
makes artifactapi cache repomd.xml forever: classifier.go checks
immutable_patterns before the RPM provider's own rule (which already classifies
repodata/ as mutable), and SetTTL with ttl=0 writes a Redis key with no expiry.
The result is a stale repomd.xml that keeps pointing at checksum-named metadata
files upstream has already deleted -> 404 on dnf metadata refresh. This broke
rpmbuilder's almalinux8 builds via the epel remote and would eventually hit any
rolling RPM remote.

Remove the repodata immutable patterns from every remote_rpm config so repodata
falls through to the provider's Mutable classification and is re-fetched every
mutable_ttl. RPMs, noarch, GPG keys and .img files stay immutable.

- Drop ".*/repodata/.*\.{xml,sqlite,yaml}.*$" from all config/remote_rpm/*.yaml
  (14 files, 20 lines).

After apply, the already-cached stale repodata entries must be evicted (they
have no expiry); that is done per-remote as a follow-up.
2026-07-18 14:12:02 +10:00
unkinben 04617699aa rpm: proxy RPM Fusion free/nonfree remotes
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
The rpmfusion-free / rpmfusion-nonfree repos have no artifactapi remote,
so nodes still point at the retired packagerepo.service.consul mirror and
dnf fails to fetch repomd.xml.

Add caching remote proxies for both trees, pointed at the upstream
download1.rpmfusion.org origin, so el nodes can install RPM Fusion
packages (ffmpeg, etc.) through artifactapi.

Claude-Session: https://claude.ai/code/session_01VZ81u2JtjvaPq8Z3EDTSMb
2026-07-18 14:03:41 +10:00
benvin 53c26160f1 Merge pull request 'Add per-distro rpm-vendor local RPM repos' (#6) from benvin/rpm-vendor into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #6
2026-07-18 11:19:38 +10:00
unkinben 7bdb30d8d4 Add per-distro rpm-vendor local RPM repos
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
rpmbuilder publishes vendored third-party RPMs (vault, consul, helm, ...)
which currently go to Gitea's RPM registry. Move them onto artifactapi so
they live in real yum repos alongside rpm-internal, and keep vendored
packages separate from in-house software. rpmbuilder targets EL8, EL9 and
Fedora 42/43/44, so each distro gets its own repo to avoid EL/Fedora RPMs
colliding in a single repodata set.

- Add config/local_rpm/rpm-vendor-el8.yaml
- Add config/local_rpm/rpm-vendor-el9.yaml
- Add config/local_rpm/rpm-vendor-f42.yaml
- Add config/local_rpm/rpm-vendor-f43.yaml
- Add config/local_rpm/rpm-vendor-f44.yaml

config.hcl already globs local_rpm/*.yaml, so no wiring change is needed.
2026-07-18 00:08:58 +10:00
benvin 834a04a034 Merge pull request 'dockerhub: proxy grafana/ images' (#5) from benvin/grafana-images into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #5
2026-07-06 23:52:55 +10:00
unkinben 7ce31f864c dockerhub: proxy grafana/ images
ci/woodpecker/pr/plan Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
The grafana-operator creates Grafana pods from docker.io/grafana/grafana,
which the dockerhub remote did not allow, so the pods would ImagePullBackOff.
Add the ^grafana/ pattern so the grafana image (and image-renderer) proxy
through artifactapi.
2026-07-06 22:11:47 +10:00
benvin 78caca1d69 Merge pull request 'feat: manage local docker registries' (#4) from benvin/local-docker into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #4
2026-07-05 22:43:12 +10:00
unkinben c42826c6de feat: manage local docker registries
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
ArtifactAPI now serves local docker repos as real container registries and the
provider exposes an artifactapi_local_docker resource, but this config had no
way to declare one.

- Add a local_docker variable, module resource, config loader wiring, and
  terragrunt input, mirroring the other local_* kinds.
- Declare a docker-internal registry.
- Expand the README to cover the local_* and virtual kinds.

Depends on a terraform-provider-artifactapi release exposing
artifactapi_local_docker (and a matching .terraform.lock.hcl bump) before apply.
2026-07-05 17:16:17 +10:00
benvin 38ab457ac1 Merge pull request 'feat: initial commit' (#3) from benvin/add-missing-registries into main
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #3
2026-07-03 23:35:30 +10:00
unkinben 9351ea30a9 fix: declare provider requirement in the artifactapi module
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
pre-commit runs `tofu validate`/tflint on modules/artifactapi standalone, where
no required_providers was declared, so OpenTofu inferred hashicorp/artifactapi
and failed (and tflint flagged missing required_version/version constraint).

- add modules/artifactapi/versions.tf declaring required_version and the
  artifactapi provider (source = the registry, version >= 0.1.2)
- drop the now-duplicate required_providers from root.hcl's generated backend.tf;
  keep the provider config block + backend. Declaring it in both the module and
  the generated root would be a "Duplicate required providers" error at runtime.

Also revert the earlier init lock-cleanup: the CI failure was stale provider
references in state (fixed with `tofu state replace-provider`), not the lock.
2026-07-03 23:34:19 +10:00
unkinben 292392a024 fix(ci): clear stale provider locks before init
ci/woodpecker/pr/pre-commit Pipeline failed
ci/woodpecker/pr/plan Pipeline failed
The provider source moved from git.unkin.net/unkin/artifactapi to
artifactapi.k8s.syd1.au.unkin.net/terraform-unkin/artifactapi. .terraform.lock.hcl
is gitignored, so a CI runner reusing a workspace/cache keeps a lock pinning the
old source; some OpenTofu versions then try to re-resolve git.unkin.net (which
isn't a registry) instead of dropping it, failing init.

Delete any .terraform.lock.hcl before `terragrunt run --all init -- -upgrade` so
providers resolve purely from config.
2026-07-03 21:48:57 +10:00
unkinben 9058eb3f59 feat: initial commit
ci/woodpecker/pr/pre-commit Pipeline failed
ci/woodpecker/pr/plan Pipeline was successful
- add modules
- add config
- add environments
- add .gitignore
- add makefile

Wire up config.hcl, variables, and terragrunt inputs for all new types.
2026-07-03 21:29:02 +10:00