Add npm virtual repositories #152

Merged

benvin merged 9 commits from benvin/virtual-npm into master 2026-10-17 21:52:56 +11:00

Conversation 2 Commits 9 Files Changed 12

+793 -12

unkin-agent commented 2026-10-17 17:00:47 +11:00

Member

Copy Link

Copy Source

npm clients need one registry for internal and public packages, but npm virtuals have no merger, so a local npm repo and npmjs cannot be served together.

• merge packuments across members: union versions (member order wins), first-defined dist-tags minus ones naming missing versions
• serve full and abbreviated (install-v1) packuments, scoped names included, via the shared virtual merge cache
• point tarballs at <virtual>/<member>/<path> and 302 them to the owning member
• reject an invalid Host with 400; serve virtual and local packuments no-store with Vary: Host, X-Forwarded-Proto, Accept
• return 502 when a member errors; skip members without the package
• add docs/npm/virtual.md and a stock node:22 e2e-docker case

npm clients need one registry for internal and public packages, but npm virtuals have no merger, so a local npm repo and npmjs cannot be served together. - merge packuments across members: union versions (member order wins), first-defined dist-tags minus ones naming missing versions - serve full and abbreviated (`install-v1`) packuments, scoped names included, via the shared virtual merge cache - point tarballs at `<virtual>/<member>/<path>` and 302 them to the owning member - reject an invalid Host with 400; serve virtual and local packuments `no-store` with `Vary: Host, X-Forwarded-Proto, Accept` - return 502 when a member errors; skip members without the package - add `docs/npm/virtual.md` and a stock `node:22` e2e-docker case

unkin-agent commented 2026-10-17 17:06:48 +11:00

Author

Member

Copy Link

Copy Source

• internal/api/v1/proxy.go:178 — proxyBaseURL is built from raw r.Host and any X-Forwarded-Proto value with no validation, then written into every rewritten tarball URL (npm_merger.go:203) and used in the cache key; a crafted Host (e.g. x@evil.com) or proto yields tarball URLs npm will fetch from an attacker host → validate host (validHost) and scheme (https/http only, shared Scheme()) before FetchNPM/MemberRedirect and return 400; both helpers live only in benvin/nuget-followups, so land/rebase on that rather than adding a second copy.
• internal/api/v1/proxy.go:209 — host-dependent, Accept-dependent body (full vs install-v1) is served with no Vary / Cache-Control → a shared cache can serve one host's rewritten URLs, or the abbreviated doc, to other clients; set Vary: Accept, Host, X-Forwarded-Proto and Cache-Control: no-store.
• internal/virtual/npm_merger.go:203 — cache key includes the unvalidated host, so a client varying Host fills the 1024-entry cap and evicts legitimate packuments (cache thrash, repeated fan-out to members) → fixed by validating the host (first finding); also consider keying on virtual+package only and rewriting the host per request.
• nit: npm_merger.go:217 — evictOldest runs on every request including cache hits, scanning all e.gens keys under the global mutex → only call it when merged inserted a new key.

- internal/api/v1/proxy.go:178 — `proxyBaseURL` is built from raw `r.Host` and any `X-Forwarded-Proto` value with no validation, then written into every rewritten tarball URL (npm_merger.go:203) and used in the cache key; a crafted Host (e.g. `x@evil.com`) or proto yields tarball URLs npm will fetch from an attacker host → validate host (`validHost`) and scheme (https/http only, shared `Scheme()`) before FetchNPM/MemberRedirect and return 400; both helpers live only in benvin/nuget-followups, so land/rebase on that rather than adding a second copy. - internal/api/v1/proxy.go:209 — host-dependent, Accept-dependent body (full vs install-v1) is served with no `Vary` / `Cache-Control` → a shared cache can serve one host's rewritten URLs, or the abbreviated doc, to other clients; set `Vary: Accept, Host, X-Forwarded-Proto` and `Cache-Control: no-store`. - internal/virtual/npm_merger.go:203 — cache key includes the unvalidated host, so a client varying Host fills the 1024-entry cap and evicts legitimate packuments (cache thrash, repeated fan-out to members) → fixed by validating the host (first finding); also consider keying on virtual+package only and rewriting the host per request. - nit: npm_merger.go:217 — `evictOldest` runs on every request including cache hits, scanning all `e.gens` keys under the global mutex → only call it when `merged` inserted a new key.

benvin changed target branch from benvin/local-npm to master 2026-10-17 17:18:07 +11:00

benvin added 7 commits 2026-10-17 17:18:07 +11:00

Add local npm registry publish and install d8e2446262

Bound npm publish body and make interrupted publishes retryable 4747693fa8

Merge remote-tracking branch 'origin/master' into benvin/local-npm 1c61162dc7

Write npm dist-tags only after the manifest lands cea1da59f6

Reject npm publish over an existing version 76d108d510

Merge remote-tracking branch 'origin/master' into benvin/local-npm ... 7d284391ca

# Conflicts:
#	e2e-docker/README.md
#	ui/src/components/UsageInstructions.tsx

Add npm virtual repositories 2c20009599

unkin-agent added 2 commits 2026-10-17 21:46:20 +11:00

Merge origin/master into benvin/virtual-npm b66b2b1567

Validate Host and mark npm packuments per-host uncacheable 8dcbe55936

unkin-agent commented 2026-10-17 21:50:14 +11:00

Author

Member

Copy Link

Copy Source

No findings.

No findings.

benvin merged commit 2c671a1be7 into master 2026-10-17 21:52:56 +11:00

benvin deleted branch benvin/virtual-npm 2026-10-17 21:52:56 +11:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/artifactapi#152