Virtual merges expire after 60s and the next request blocks on every member. One slow member (a large github_rpm remote) makes the re-merge outlast dnf's 30s timeout.
- add `serve_stale` to remotes (model, v2 API, migration 0008, UI, TUI), default false
- serve a flagged member's last good fetch to rpm/deb/alpine merges at once
- refresh it in the background, single-flighted, detached from the request
- block on the member again once its copy is over an hour old, or the flag is off
- prune copies with their merge cache; drop refreshes for evicted entries
Reviewed-on: #167
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Remotes store `quarantine_enabled`/`quarantine_days` but the proxy never enforces them, so freshly published (possibly malicious) artifacts are cached immediately.
- refuse uncached immutable paths published under `quarantine_days` ago with a JSON 403
- take publish time from `Last-Modified` capped at now, else a Redis first-seen key; 503 + `Retry-After` when that lookup fails
- reject enabled quarantine with `quarantine_days<1`; backfill stored `<=0` to 3
- default `quarantine_days` to 3 on create and keep it on update
- let goproxy virtual probes skip a quarantining member
Reviewed-on: #170
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Virtual members already merge concurrently, but each rpm member fetches primary/filelists/other one after another, and each deb member its Packages files the same way. On large virtuals the slowest member's serial chain pushes the merge past dnf's 30s timeout.
- add `parallel_fetch` (default false) to virtuals: model, v2 API, migration 0007
- fetch a member's index files concurrently (up to 4) when set; first error still fails the member
- thread the flag through rpm and deb member fetches
Reviewed-on: #165
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
A GitHub release scan that hit a transient asset error still saved the release-list ETag, so the failed asset was never retried. Assets that are corrupt, deleted or truncated upstream fail on every retry, so they must not keep the remote failed.
- fail the rpm/deb/alpine scan on transient asset fetch, short read or metadata insert errors, so the ETag is not saved and the backoff retry re-derives the asset
- add `provider.ErrInvalidPackage` for unparseable packages and 404/410/416 asset downloads; such assets are logged and skipped
- decide a ranged fetch holds the whole object from Content-Length or the Content-Range total, or a clean EOF when the total is unknown
- add a NOT VALID cascading FK from the GitHub sync-state tables to `remotes`
Reviewed-on: #135
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
In-house charts have nowhere to publish: a helm local stores uploads but serves no index.yaml, and the helm virtual merger drops every chart field except name/version/urls.
- add helm local uploads (`PUT .../files/<name>-<version>.tgz`) recording each Chart.yaml in a new `helm_metadata` table; mismatched filenames stay out of the index
- bound chart parsing (50MiB decompressed, 10k entries, 1MiB Chart.yaml)
- generate a deterministic `index.yaml` with sha256 digests and path-escaped relative urls
- keep all chart fields when merging helm virtual indexes, expanding YAML aliases
- add a real-helm e2e-docker test, UI snippet and README section
Reviewed-on: #140
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Puppet modules can only be proxied from a remote Forge; there is nowhere to publish internal modules for puppet module install, r10k and g10k.
- add puppet local uploads at <owner>-<module>-<version>.tar.gz, rejecting existing versions
- record each release's metadata.json, md5 and sha256 in a puppet_metadata table, streaming the tarball under size caps
- serve Forge v3 /v3/modules, paginated /v3/releases, /v3/releases/<slug> and /v3/files
- add an e2e-docker case driving real r10k and puppet module install
- document local puppet in README and UI usage snippets
Reviewed-on: #145
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Releasing a GitHub sync lease always advanced `last_synced_at`, even after a failed scan (e.g. a rate-limit 403). A remote that failed once waited a full `mutable_ttl` before retrying, and a cold remote kept returning 503 until then.
- record scan outcomes in one shared lease helper for github_rpm/deb/alpine
- keep `last_synced_at` and the ETag on failure; retry from 60s with exponential backoff, capped at min(10m, ttl/4)
- honour `Retry-After` / `X-RateLimit-Reset`, clamped to `mutable_ttl`
- add `sync_failures` / `next_retry_at` columns (migration 0002)
Reviewed-on: #133
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
The Postgres schema was a ~180-line DDL blob inlined in `internal/database/postgres.go` and re-executed on every start. It could only grow — each change appended another `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` — and nothing recorded what had ever been applied. `golib/pg` already owns that mechanic for the estate (arrproxy#10), so artifactapi keeps the SQL and hands over the machinery.
## How
- Depend on `git.unkin.net/unkin/golib` v0.1.0.
- Move the DDL **verbatim** into `migrations/0001_init.sql`, embedded by the new `migrations` package, and build the pool with `pg.NewMigrated` (`LockName: "artifactapi-migrations"`). The runner takes a cluster-wide advisory lock the old blob never took, so replicas starting together queue instead of racing each other through the DDL.
- **No-op on the live database.** Production has the schema but no `schema_migrations`, so its first start on this build re-runs `0001`. Every statement is `IF NOT EXISTS`-guarded, so that run changes nothing and only lands the tracking row. `TestMigratingAnAlreadyPopulatedSchemaIsANoOp` drops the row from a migrated database and asserts exactly that; `TestMigrationsAreIdempotent` keeps future migrations additive and idempotent.
- **Env var contract unchanged.** `config.DatabaseDSN` stays the DSN source rather than `pg.DSNFromEnv`: the variable names match, but golib deliberately has no default user or database name, and artifactapi documents and ships `DBUSER`/`DBNAME` defaults of `artifacts`.
- Drift guards: the embedded set is asserted equal to `migrations/`, and the derived advisory key is pinned so a rename cannot silently let two builds migrate at once.
- Plumb `GOPRIVATE=git.unkin.net` for the first cross-repo Go dependency — exported by the `Makefile`, set in the `Dockerfile` and both woodpecker Go steps, documented in the README.
`database.New`'s signature and every caller are untouched.
Transitive bumps come with golib's `go.mod` (testcontainers-go 0.42 → 0.44, otel, x/crypto, x/net); `go build`, `go vet`, `golangci-lint`, `go test -race ./pkg/... ./internal/...` and `pre-commit run --all-files` are clean.
Reviewed-on: #125
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
unkin-agent