Add local npm registry publish and install #139

Open

unkin-agent wants to merge 5 commits from benvin/local-npm into master

pull from: benvin/local-npm

merge into: unkin:master

unkin:master

unkin:benvin/local-helm

unkin:benvin/local-go

unkin:benvin/remote-head-route

unkin:benvin/nuget-followups

unkin:benvin/virtual-deb

unkin:benvin/remote-ansible

unkin:benvin/github-asset-fail-etag

unkin:benvin/remote-query-forwarder

unkin:benvin/local-cargo

unkin:benvin/auth-design-doc

unkin:benvin/yaml_updates

unkin:benvin/develop

Conversation 3 Commits 5 Files Changed 8

+891 -1

unkin-agent commented 2026-10-16 23:08:42 +11:00

Member

Copy Link

Copy Source

In-house npm packages have nowhere to publish: npm locals only accept raw file uploads, which npm publish and npm install cannot use.

• accept npm publish (PUT /api/v1/local/<repo>/<pkg>, scoped included) on local npm repos, capping the body at 256MiB and verifying the tarball against dist integrity
• write tarball, then manifest, then dist-tags; publishing an existing version returns 409 and leaves tags alone, while a matching leftover tarball from a failed publish is reused so retries complete
• serve packuments with tarball URLs pointing back at the local repo, dropping dist-tags that name unpublished versions
• document the .npmrc registry + _authToken setup in the UI and README
• add a stock node:22 publish/install e2e-docker case

In-house npm packages have nowhere to publish: npm locals only accept raw file uploads, which `npm publish` and `npm install` cannot use. - accept `npm publish` (`PUT /api/v1/local/<repo>/<pkg>`, scoped included) on local npm repos, capping the body at 256MiB and verifying the tarball against dist integrity - write tarball, then manifest, then dist-tags; publishing an existing version returns 409 and leaves tags alone, while a matching leftover tarball from a failed publish is reused so retries complete - serve packuments with tarball URLs pointing back at the local repo, dropping dist-tags that name unpublished versions - document the `.npmrc` registry + `_authToken` setup in the UI and README - add a stock `node:22` publish/install e2e-docker case

unkin-agent added 1 commit 2026-10-16 23:08:42 +11:00

Add local npm registry publish and install d8e2446262

unkin-agent referenced this pull request from unkin/terraform-provider-artifactapi 2026-10-16 23:10:49 +11:00

Add artifactapi_local_npm resource #24

unkin-agent commented 2026-10-16 23:18:42 +11:00

Author

Member

Copy Link

Copy Source

• internal/api/v1/npm.go:75 — publish body decoded with no size bound (JSON + base64 string + decoded tarball all held in memory) → wrap r.Body in http.MaxBytesReader with a configurable cap, return 413 on overflow.
• internal/api/v1/npm.go:171 — tarball row is committed before the manifest/dist-tag rows; a failure on the manifest or tag write leaves the version invisible in the packument yet permanently 409 on retry → make the publish resumable (on ErrAlreadyExists with no manifest row, finish the remaining writes) or write the gate last.
• internal/api/v1/npm_test.go:1 — no test drives handleLocalPut/handleLocal (409 on republish, 405 on non-npm local, scoped %2f through chi routing, two concurrent publishes of different versions keeping both versions and tags, failure after tarball write) → add handler-level tests with a fake db/cas; concurrency and immutability are only covered by the docker e2e.
• nit: internal/api/v1/npm.go:89 — npmRefRe accepts non-semver versions (e.g. 1..2, x+) that npm clients choke on in a packument → validate with a semver parser.
• nit: internal/api/v1/npm.go:220 — every packument GET reads every version manifest blob from storage → fine now; note the O(versions) S3 reads or store a per-package index.

- internal/api/v1/npm.go:75 — publish body decoded with no size bound (JSON + base64 string + decoded tarball all held in memory) → wrap `r.Body` in `http.MaxBytesReader` with a configurable cap, return 413 on overflow. - internal/api/v1/npm.go:171 — tarball row is committed before the manifest/dist-tag rows; a failure on the manifest or tag write leaves the version invisible in the packument yet permanently 409 on retry → make the publish resumable (on `ErrAlreadyExists` with no manifest row, finish the remaining writes) or write the gate last. - internal/api/v1/npm_test.go:1 — no test drives `handleLocalPut`/`handleLocal` (409 on republish, 405 on non-npm local, scoped `%2f` through chi routing, two concurrent publishes of different versions keeping both versions and tags, failure after tarball write) → add handler-level tests with a fake db/cas; concurrency and immutability are only covered by the docker e2e. - nit: internal/api/v1/npm.go:89 — `npmRefRe` accepts non-semver versions (e.g. `1..2`, `x+`) that npm clients choke on in a packument → validate with a semver parser. - nit: internal/api/v1/npm.go:220 — every packument GET reads every version manifest blob from storage → fine now; note the O(versions) S3 reads or store a per-package index.

unkin-agent added 1 commit 2026-10-17 12:34:23 +11:00

Bound npm publish body and make interrupted publishes retryable 4747693fa8

unkin-agent commented 2026-10-17 12:38:13 +11:00

Author

Member

Copy Link

Copy Source

• internal/api/v1/npm.go:200 — dist-tags are upserted before the manifest exists, and buildNPMPackument (npm.go:307-310) emits tags unfiltered. While publishing 1.0.1 over an existing 1.0.0, the packument returns latest: 1.0.1 with no 1.0.1 in versions, so npm install pkg fails ETARGET until the manifest lands (permanently if the publish dies after the tags) → drop tags whose version is absent from versions in buildNPMPackument, and add a test for the window.
• internal/api/v1/npm.go:200 — a publish that ends in 409 has already moved the tags. Re-PUT 1.0.0 (same tarball, any manifest bytes) after 1.0.1 passes the tarball check, upserts latest -> 1.0.0, then 409s on the manifest, so latest regresses. Same via concurrent publishes with the same tarball and different manifests: the loser's tags can land last → check the manifest path first (409 before touching tags), or only write tags once the manifest write has succeeded; test a rejected republish leaves tags unchanged.
• nit: internal/api/v1/npm.go:206 — the manifest is not byte-compared the way the tarball is, so an identical retry after a lost 201, or the loser of two concurrent identical publishes, gets 409 although the data is identical → on ErrAlreadyExists compare the stored hash and return 201 when equal.

- internal/api/v1/npm.go:200 — dist-tags are upserted before the manifest exists, and buildNPMPackument (npm.go:307-310) emits tags unfiltered. While publishing 1.0.1 over an existing 1.0.0, the packument returns `latest: 1.0.1` with no `1.0.1` in `versions`, so `npm install pkg` fails ETARGET until the manifest lands (permanently if the publish dies after the tags) → drop tags whose version is absent from `versions` in buildNPMPackument, and add a test for the window. - internal/api/v1/npm.go:200 — a publish that ends in 409 has already moved the tags. Re-PUT 1.0.0 (same tarball, any manifest bytes) after 1.0.1 passes the tarball check, upserts `latest -> 1.0.0`, then 409s on the manifest, so `latest` regresses. Same via concurrent publishes with the same tarball and different manifests: the loser's tags can land last → check the manifest path first (409 before touching tags), or only write tags once the manifest write has succeeded; test a rejected republish leaves tags unchanged. - nit: internal/api/v1/npm.go:206 — the manifest is not byte-compared the way the tarball is, so an identical retry after a lost 201, or the loser of two concurrent identical publishes, gets 409 although the data is identical → on ErrAlreadyExists compare the stored hash and return 201 when equal.

unkin-agent added 2 commits 2026-10-17 12:44:42 +11:00

Merge remote-tracking branch 'origin/master' into benvin/local-npm 1c61162dc7

Write npm dist-tags only after the manifest lands cea1da59f6

unkin-agent added 1 commit 2026-10-17 12:51:58 +11:00

Reject npm publish over an existing version 76d108d510

unkin-agent commented 2026-10-17 12:56:07 +11:00

Author

Member

Copy Link

Copy Source

No findings.

No findings.

All checks were successful

Hide all checks

ci/woodpecker/pr/pre-commit Pipeline was successful

Required

Details

ci/woodpecker/pr/test Pipeline was successful

Required

Details

ci/woodpecker/pr/build Pipeline was successful

Required

Details

This pull request has changes conflicting with the target branch.

• e2e-docker/README.md

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin benvin/local-npm:benvin/local-npm

git checkout benvin/local-npm

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/artifactapi#139