Add ansible Galaxy remote type #137

Merged

benvin merged 12 commits from benvin/remote-ansible into master 2026-10-18 12:08:55 +11:00

Conversation 3 Commits 12 Files Changed 18

+488 -4

unkin-agent commented 2026-10-16 23:07:03 +11:00

Member

Copy Link

Copy Source

ansible-galaxy installs pull collections straight from galaxy.ansible.com because artifactapi has no Galaxy remote type.

• add ansible package type proxying the Galaxy v3 API (discovery, collection versions, artifact downloads)
• forward digits-only limit (capped at 100) and offset on metadata paths, canonicalised; tarball paths forward no query
• rewrite only href, download_url and links.* to root-relative remote paths so tarballs flow through artifactapi
• treat artifacts/*.tar.gz as immutable and JSON metadata as mutable
• add docs/ansible/remote.md, a UI type def and an ansible e2e caching case

requires unkin/artifactapi#146

ansible-galaxy installs pull collections straight from galaxy.ansible.com because artifactapi has no Galaxy remote type. - add `ansible` package type proxying the Galaxy v3 API (discovery, collection versions, artifact downloads) - forward digits-only `limit` (capped at 100) and `offset` on metadata paths, canonicalised; tarball paths forward no query - rewrite only `href`, `download_url` and `links.*` to root-relative remote paths so tarballs flow through artifactapi - treat `artifacts/*.tar.gz` as immutable and JSON metadata as mutable - add `docs/ansible/remote.md`, a UI type def and an ansible e2e caching case requires unkin/artifactapi#146

unkin-agent referenced this pull request from unkin/terraform-provider-artifactapi 2026-10-16 23:08:32 +11:00

resource_remote: add artifactapi_remote_ansible #23

unkin-agent commented 2026-10-16 23:21:40 +11:00

Author

Member

Copy Link

Copy Source

• internal/api/v1/proxy.go:95 — raw client query is appended to the path, so it becomes the cache key (Redis TTL, artifacts row, S3 indexes/<remote>/<path>?<q>) and is sent upstream verbatim: any ?x=<random> mints unlimited cache entries/objects and upstream requests, and param order/encoding variants split keys → forward only an allowlist (limit, offset; add others only if needed), re-encode via url.Values.Encode() (sorted), cap value length/digits, drop everything else.
• internal/provider/ansible/ansible.go:63 — isArtifact runs on path+query, so .../versions/?x=.tar.gz is classified Immutable (cached forever, never rewritten) and content-typed gzip → strip the query (path, _, _ = strings.Cut(path, "?")) before the suffix check; also require the artifacts path segment, not just .tar.gz.
• internal/api/v1/proxy.go:95 — query is appended for every ansible path including tarballs, so ?n=1..N on one tarball creates N artifact rows/aliases for the same bytes → only forward the query on non-artifact (metadata) paths.
• internal/provider/ansible/ansible.go:53 — text-level ReplaceAll of "/api/ rewrites any JSON string value starting with /api/ (descriptions, readme fields) and hardcodes the /api/v1/remote/ mount; remote.Name is spliced into the body unescaped → unmarshal and rewrite only href, download_url, and links.*, or at minimum JSON-escape the prefix.
• internal/provider/ansible/ansible_test.go:81 — TestForwardsQuery only asserts the marker interface; nothing tests the proxy.go change (query appended, no query = unchanged path, non-forwarder providers unchanged) or the cache key → add a handler-level test with a forwarder and a non-forwarder, and a test that unknown params are dropped.
• internal/provider/ansible/ansible_test.go:12 — Classify has no case with a query suffix (see .tar.gz finding) and rewrite has no case for a body with other /api/ strings or odd remote names → add them.
• e2e-docker/caching_test.go:33 — only the tarball is exercised; the metadata path (paginated versions/?limit=&offset= + rewritten links) the PR exists for has no e2e → add a case following links.next through the proxy.
• scope — QueryForwarder is a cross-cutting change to the shared proxy handler/cache-key semantics bundled with a new provider → split into PRs: (1) provider.QueryForwarder + proxy.go + allowlist + tests, (2) ansible provider/model/server registration, (3) UI + README + e2e (or fold into 2).

- internal/api/v1/proxy.go:95 — raw client query is appended to the path, so it becomes the cache key (Redis TTL, artifacts row, S3 `indexes/<remote>/<path>?<q>`) and is sent upstream verbatim: any `?x=<random>` mints unlimited cache entries/objects and upstream requests, and param order/encoding variants split keys → forward only an allowlist (`limit`, `offset`; add others only if needed), re-encode via `url.Values.Encode()` (sorted), cap value length/digits, drop everything else. - internal/provider/ansible/ansible.go:63 — `isArtifact` runs on path+query, so `.../versions/?x=.tar.gz` is classified Immutable (cached forever, never rewritten) and content-typed gzip → strip the query (`path, _, _ = strings.Cut(path, "?")`) before the suffix check; also require the artifacts path segment, not just `.tar.gz`. - internal/api/v1/proxy.go:95 — query is appended for every ansible path including tarballs, so `?n=1..N` on one tarball creates N artifact rows/aliases for the same bytes → only forward the query on non-artifact (metadata) paths. - internal/provider/ansible/ansible.go:53 — text-level `ReplaceAll` of `"/api/` rewrites any JSON string value starting with `/api/` (descriptions, readme fields) and hardcodes the `/api/v1/remote/` mount; remote.Name is spliced into the body unescaped → unmarshal and rewrite only `href`, `download_url`, and `links.*`, or at minimum JSON-escape the prefix. - internal/provider/ansible/ansible_test.go:81 — `TestForwardsQuery` only asserts the marker interface; nothing tests the proxy.go change (query appended, no query = unchanged path, non-forwarder providers unchanged) or the cache key → add a handler-level test with a forwarder and a non-forwarder, and a test that unknown params are dropped. - internal/provider/ansible/ansible_test.go:12 — Classify has no case with a query suffix (see `.tar.gz` finding) and rewrite has no case for a body with other `/api/` strings or odd remote names → add them. - e2e-docker/caching_test.go:33 — only the tarball is exercised; the metadata path (paginated `versions/?limit=&offset=` + rewritten links) the PR exists for has no e2e → add a case following `links.next` through the proxy. - scope — QueryForwarder is a cross-cutting change to the shared proxy handler/cache-key semantics bundled with a new provider → split into PRs: (1) `provider.QueryForwarder` + proxy.go + allowlist + tests, (2) ansible provider/model/server registration, (3) UI + README + e2e (or fold into 2).

unkin-agent commented 2026-10-17 12:48:52 +11:00

Author

Member

Copy Link

Copy Source

No findings.

No findings.

unkin-agent changed target branch from master to benvin/remote-query-forwarder 2026-10-17 12:52:24 +11:00

unkin-agent referenced this pull request 2026-10-18 03:38:04 +11:00

Add local ansible Galaxy repositories #157

unkin-agent commented 2026-10-18 03:53:37 +11:00

Author

Member

Copy Link

Copy Source

No findings.

No findings.

benvin changed target branch from benvin/remote-query-forwarder to master 2026-10-18 12:08:23 +11:00

benvin added 12 commits 2026-10-18 12:08:23 +11:00

Add ansible Galaxy remote type ... 13d693b58a

- add ansible package type proxying the Galaxy v3 API
- rewrite href/links/download_url back to the remote so tarballs flow through it
- forward the query string for providers that opt in, keeping Galaxy pagination working
- add a UI usage snippet and an ansible e2e caching case

Retrigger CI 0f826aa331

Forward allowlisted query params for opt-in remote providers 586b07bab6

Merge benvin/remote-query-forwarder into benvin/remote-ansible 7bb11b4761

Forward ansible pagination params and rewrite only Galaxy link fields af6a088ef4

Document ansible pagination and artifact paths 9935691c6b

Classify bare paths, validate forwarded query values, evict query variants 769d673a4a

Merge remote-tracking branch 'origin/benvin/remote-query-forwarder' into benvin/remote-ansible ef5a38475f

Validate and cap ansible pagination params 285e8674ca

Merge master into benvin/remote-query-forwarder 71e6458c77

Merge benvin/remote-query-forwarder into benvin/remote-ansible f4d3093952

List ansible in the per-type caching docs 583a3ad4d1

benvin merged commit ca08c580b8 into master 2026-10-18 12:08:55 +11:00

benvin deleted branch benvin/remote-ansible 2026-10-18 12:08:55 +11:00

benvin referenced this issue from a commit 2026-10-18 18:01:10 +11:00

Add local ansible Galaxy repositories (#157)

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/artifactapi#137