Add helm local repositories #140

Merged

benvin merged 7 commits from benvin/local-helm into master 2026-10-17 15:14:36 +11:00

Conversation 3 Commits 7 Files Changed 11

+863 -43

unkin-agent commented 2026-10-16 23:12:48 +11:00

Member

Copy Link

Copy Source

In-house charts have nowhere to publish: a helm local stores uploads but serves no index.yaml, and the helm virtual merger drops every chart field except name/version/urls.

• add helm local uploads (PUT .../files/<name>-<version>.tgz) recording each Chart.yaml in a new helm_metadata table; mismatched filenames stay out of the index
• bound chart parsing (50MiB decompressed, 10k entries, 1MiB Chart.yaml)
• generate a deterministic index.yaml with sha256 digests and path-escaped relative urls
• keep all chart fields when merging helm virtual indexes, expanding YAML aliases
• add a real-helm e2e-docker test, UI snippet and README section

In-house charts have nowhere to publish: a helm local stores uploads but serves no index.yaml, and the helm virtual merger drops every chart field except name/version/urls. - add helm local uploads (`PUT .../files/<name>-<version>.tgz`) recording each Chart.yaml in a new `helm_metadata` table; mismatched filenames stay out of the index - bound chart parsing (50MiB decompressed, 10k entries, 1MiB Chart.yaml) - generate a deterministic `index.yaml` with sha256 digests and path-escaped relative urls - keep all chart fields when merging helm virtual indexes, expanding YAML aliases - add a real-helm e2e-docker test, UI snippet and README section

unkin-agent added 1 commit 2026-10-16 23:12:48 +11:00

Add helm local repositories 18e7f000f7

unkin-agent referenced this pull request from unkin/terraform-provider-artifactapi 2026-10-16 23:17:15 +11:00

Add artifactapi_local_helm resource #26

unkin-agent added 1 commit 2026-10-16 23:18:33 +11:00

Retrigger CI dbb1af26d5

unkin-agent added 1 commit 2026-10-16 23:25:45 +11:00

Retrigger CI 7539f93c01

unkin-agent commented 2026-10-17 12:31:39 +11:00

Author

Member

Copy Link

Copy Source

• internal/provider/helm/helm.go:206 — parseChart walks the whole gzip/tar stream with no cap on decompressed bytes or entry count; a Chart.yaml placed last in a small high-ratio archive makes tr.Next() decompress GBs in a goroutine with context.Background() → bound the gzip reader (e.g. io.LimitReader at a few hundred MiB) and cap entries scanned.
• internal/provider/helm/helm.go:73 — storage filename is never checked against the chart name/version, so two differently named tgz of the same chart produce duplicate index entries, and a filename with space/#/? is emitted unescaped as a relative url → require <name>-<version>.tgz after parse (or dedupe by name+version) and restrict the filename charset.
• internal/provider/helm/helm.go:217 — LimitReader silently truncates Chart.yaml at 1 MiB and stores the truncated text → read limit+1 and reject when exceeded.
• internal/virtual/helm_merger.go:53 — chart fields are now raw yaml.Node; if a remote member index uses anchors (&a) and the anchor-bearing entry is deduped by version, the surviving *a alias is emitted dangling and the merged index is invalid YAML → add a test with an anchored remote entry and resolve aliases before merging.
• nit: migrations/0003_helm_metadata.sql — no collision with master (0001, 0002), but benvin/local-cargo (0003_cargo_index) and benvin/github-asset-fail-etag (0003_github_sync_state_cascade) share the 0003 prefix; no test enforces unique prefixes and versions are tracked by full filename so they coexist → renumber whichever merges second.

- internal/provider/helm/helm.go:206 — parseChart walks the whole gzip/tar stream with no cap on decompressed bytes or entry count; a Chart.yaml placed last in a small high-ratio archive makes tr.Next() decompress GBs in a goroutine with context.Background() → bound the gzip reader (e.g. io.LimitReader at a few hundred MiB) and cap entries scanned. - internal/provider/helm/helm.go:73 — storage filename is never checked against the chart name/version, so two differently named tgz of the same chart produce duplicate index entries, and a filename with space/#/? is emitted unescaped as a relative url → require `<name>-<version>.tgz` after parse (or dedupe by name+version) and restrict the filename charset. - internal/provider/helm/helm.go:217 — LimitReader silently truncates Chart.yaml at 1 MiB and stores the truncated text → read limit+1 and reject when exceeded. - internal/virtual/helm_merger.go:53 — chart fields are now raw yaml.Node; if a remote member index uses anchors (`&a`) and the anchor-bearing entry is deduped by version, the surviving `*a` alias is emitted dangling and the merged index is invalid YAML → add a test with an anchored remote entry and resolve aliases before merging. - nit: migrations/0003_helm_metadata.sql — no collision with master (0001, 0002), but benvin/local-cargo (0003_cargo_index) and benvin/github-asset-fail-etag (0003_github_sync_state_cascade) share the 0003 prefix; no test enforces unique prefixes and versions are tracked by full filename so they coexist → renumber whichever merges second.

unkin-agent referenced this pull request 2026-10-17 12:31:56 +11:00

Add local cargo registry type #144

unkin-agent added 2 commits 2026-10-17 12:37:39 +11:00

Merge remote-tracking branch 'origin/master' into benvin/local-helm 58be0721c8

Bound chart parsing, require name-version filenames, expand index aliases 5280817d80

unkin-agent commented 2026-10-17 12:45:19 +11:00

Author

Member

Copy Link

Copy Source

• internal/virtual/helm_merger.go:41 — budget counts every node, not just alias expansion; a legitimate 10k-version index (~100 nodes/entry... measured 100 charts x 100 versions) is silently truncated to ~5.5k versions, and which charts survive depends on map iteration order (nondeterministic merged index) → count only nodes reached through an alias (or raise the cap well above real indexes) and do not silently drop entries.
• internal/provider/helm/helm.go:147 — a Chart.yaml anchor on a key that is then overwritten (version: &v 1.0.0 + appVersion: *v, same for name) leaves a dangling *v in the generated index.yaml; the output fails to parse ("unknown anchor") and breaks the whole repo index → expand aliases/clear anchors in chart (as the merger does) or re-encode the Chart.yaml to plain values before overriding fields; add a test.
• nit: internal/virtual/helm_merger.go:57 — cycle/overflow expandAliases error drops the entry with no log → slog.Warn with chart name.

- internal/virtual/helm_merger.go:41 — `budget` counts every node, not just alias expansion; a legitimate 10k-version index (~100 nodes/entry... measured 100 charts x 100 versions) is silently truncated to ~5.5k versions, and which charts survive depends on map iteration order (nondeterministic merged index) → count only nodes reached through an alias (or raise the cap well above real indexes) and do not silently drop entries. - internal/provider/helm/helm.go:147 — a Chart.yaml anchor on a key that is then overwritten (`version: &v 1.0.0` + `appVersion: *v`, same for `name`) leaves a dangling `*v` in the generated index.yaml; the output fails to parse ("unknown anchor") and breaks the whole repo index → expand aliases/clear anchors in `chart` (as the merger does) or re-encode the Chart.yaml to plain values before overriding fields; add a test. - nit: internal/virtual/helm_merger.go:57 — cycle/overflow `expandAliases` error drops the entry with no log → slog.Warn with chart name.

unkin-agent added 1 commit 2026-10-17 12:52:38 +11:00

Count only alias-reached nodes and expand chart aliases before index overrides 1ac8f60196

unkin-agent commented 2026-10-17 12:56:56 +11:00

Author

Member

Copy Link

Copy Source

• internal/provider/helm/helm.go:174 (generateIndex), internal/virtual/helm_merger.go:44 — one shared alias budget per index/member is never reset and stays negative after an overrun, so a single bomb chart makes every later alias-using chart get dropped (and a near-limit one starves them) → reset the budget per chart (generateIndex: per meta; merger: per chart name) so a bad entry only drops itself.

- internal/provider/helm/helm.go:174 (generateIndex), internal/virtual/helm_merger.go:44 — one shared alias budget per index/member is never reset and stays negative after an overrun, so a single bomb chart makes every later alias-using chart get dropped (and a near-limit one starves them) → reset the budget per chart (generateIndex: per meta; merger: per chart name) so a bad entry only drops itself.

unkin-agent added 1 commit 2026-10-17 15:11:19 +11:00

Merge origin/master into benvin/local-helm 37f8a40f87

benvin merged commit aaa923ffd1 into master 2026-10-17 15:14:36 +11:00

benvin deleted branch benvin/local-helm 2026-10-17 15:14:36 +11:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/artifactapi#140