split per-type registration points into per-type files
- move README per-type sections into docs/<type>/{remote,local,virtual}.md
- split provider.go capability and metadata types into cap_*.go and metadata_*.go
- register providers via internal/provider/all, one file per type
- declare each package type in its own file, registering via init()
- split the e2e caching table and UI type colours/snippets per type
This commit is contained in:
@@ -20,98 +20,22 @@ API: `http://localhost:8000` | Frontend: `http://localhost:5173`
|
||||
|
||||
## Package Types
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `generic` | nothing | everything |
|
||||
| `docker` | tag manifests, `/tags/list` | blobs, digest manifests |
|
||||
| `helm` | `index.yaml` | `.tgz` charts |
|
||||
| `pypi` | `simple/*` index pages | `.whl`, `.tar.gz` |
|
||||
| `npm` | package metadata | `.tgz` tarballs |
|
||||
| `rpm` | `repomd.xml`, `repodata/*` | `.rpm` |
|
||||
| `alpine` | `APKINDEX.tar.gz` | `.apk` |
|
||||
| `puppet` | `v3/modules/*`, `v3/releases*` | `.tar.gz` |
|
||||
| `terraform` | `*/versions` | `*/download/*/*` |
|
||||
| `goproxy` | `@v/list`, `@latest` | `.info`, `.mod`, `.zip` |
|
||||
| `cargo` | sparse index files (`config.json` synthesized) | `crates/*/*.crate` |
|
||||
| `nuget` | `index.json` (service index), registration and flat container indexes | `.nupkg`, `.nuspec` |
|
||||
| `github_rpm` | `repodata/*` (synthesized) | `.rpm` (redirected) |
|
||||
|
||||
Providers classify paths automatically. Users only configure what to proxy and TTLs.
|
||||
Each type's paths, Terraform examples and local/virtual behaviour are in `docs/<type>/`:
|
||||
|
||||
### `github_rpm` — GitHub releases as a yum repo (metadata-only, no precache)
|
||||
|
||||
A `github_rpm` remote turns a GitHub repo's **releases** into a real `dnf`/`yum`
|
||||
repository without ever caching the packages. It scans releases for `.rpm`
|
||||
assets, derives each package's metadata (NEVRA, requires/provides/conflicts/
|
||||
obsoletes, files, checksum) and **synthesizes `repodata/` on the fly**. Package
|
||||
metadata comes from a **ranged GET of just the RPM header** (the header sits at
|
||||
the front of the file, so the whole package is never downloaded); the sha256
|
||||
checksum comes from the GitHub asset `digest` when present, else a one-time
|
||||
lazy stream. Derived metadata is cached (keyed by asset) so repodata generation
|
||||
is served from primed DB rows, never a cold on-demand derive.
|
||||
|
||||
Each package's `<location>` points back at the remote, which **302-redirects**
|
||||
the download to the `releases_remote` — an existing generic `github.com` remote
|
||||
that streams the actual bytes. `dnf` follows the redirect transparently.
|
||||
|
||||
#### Background syncer
|
||||
|
||||
A single process-wide **background syncer** keeps every `github_rpm` remote's
|
||||
derived metadata current off the client request path:
|
||||
|
||||
- **Prime on create.** Creating a `github_rpm` remote enqueues a background prime
|
||||
scan, so its metadata is derived right away without blocking the create call.
|
||||
The first `dnf` request is served from cache. If a request arrives before the
|
||||
prime lands, it returns a retryable `503` (with `Retry-After`) rather than
|
||||
serving an empty repo or blocking on a multi-minute derive.
|
||||
- **Periodic re-check, driven by `mutable_ttl`.** Each remote is re-checked for
|
||||
new or changed releases no more often than its `mutable_ttl`. New/changed
|
||||
assets are derived incrementally; assets already cached are never re-fetched,
|
||||
and assets that disappear upstream are pruned.
|
||||
- **ETag / 304 conditional requests.** The releases-list `ETag` is stored per
|
||||
remote and sent as `If-None-Match`; a `304 Not Modified` means nothing changed
|
||||
and the syncer derives nothing. GitHub does not count `304` conditional
|
||||
responses against the rate limit, so an unchanged repo is nearly free — this is
|
||||
the main lever keeping GitHub traffic low.
|
||||
- **Global rate limit.** Every GitHub call (releases list + each ranged asset
|
||||
header GET) passes through a single token-bucket limiter **shared across all
|
||||
remotes**, so GitHub is never hammered. Configure a token (`password`) on the
|
||||
remote for the higher authenticated rate limit (~5000/hr vs ~60/hr
|
||||
unauthenticated).
|
||||
- **Multi-replica coordination.** State is shared through the database. Before a
|
||||
periodic scan a replica must atomically claim a per-remote lease
|
||||
(`github_rpm_sync_state`: `last_synced_at`, `etag`, `sync_lease_owner`,
|
||||
`sync_lease_expires`); only the winner scans. This bounds total GitHub load to
|
||||
~once per `mutable_ttl` regardless of replica count, and the shared `etag`
|
||||
lets any replica issue the conditional request.
|
||||
|
||||
```hcl
|
||||
# Backend that serves the actual .rpm bytes from github.com.
|
||||
resource "artifactapi_remote_generic" "github" {
|
||||
name = "github"
|
||||
base_url = "https://github.com"
|
||||
patterns = [
|
||||
"acme/tools/releases/download/.*\\.rpm$", # allowlist the repo's release assets
|
||||
]
|
||||
}
|
||||
|
||||
resource "artifactapi_remote_github_rpm" "acme-tools" {
|
||||
name = "acme-tools"
|
||||
base_url = "https://api.github.com/repos/acme/tools" # the releases API root
|
||||
releases_remote = "github" # backend for downloads
|
||||
mutable_ttl = 3600 # release re-scan interval
|
||||
|
||||
# Optional: restrict which release assets become packages (regex on filename).
|
||||
patterns = [".*\\.x86_64\\.rpm$", ".*\\.noarch\\.rpm$"]
|
||||
|
||||
# Optional: a token for private repos / higher API rate limits.
|
||||
# password = "ghp_..."
|
||||
}
|
||||
```
|
||||
|
||||
`dnf` config: `baseurl=https://artifactapi.example/api/v1/remote/acme-tools`.
|
||||
The repo is multi-arch (no `$basearch` needed) — `dnf` selects matching packages
|
||||
from the synthesized metadata.
|
||||
- `alpine`: [remote](docs/alpine/remote.md), [remote-github](docs/alpine/remote-github.md)
|
||||
- `cargo`: [remote](docs/cargo/remote.md)
|
||||
- `deb`: [remote](docs/deb/remote.md), [remote-github](docs/deb/remote-github.md)
|
||||
- `docker`: [remote](docs/docker/remote.md), [local](docs/docker/local.md)
|
||||
- `generic`: [remote](docs/generic/remote.md)
|
||||
- `goproxy`: [remote](docs/goproxy/remote.md)
|
||||
- `helm`: [remote](docs/helm/remote.md), [local](docs/helm/local.md), [virtual](docs/helm/virtual.md)
|
||||
- `npm`: [remote](docs/npm/remote.md)
|
||||
- `nuget`: [remote](docs/nuget/remote.md)
|
||||
- `puppet`: [remote](docs/puppet/remote.md), [local](docs/puppet/local.md)
|
||||
- `pypi`: [remote](docs/pypi/remote.md)
|
||||
- `rpm`: [remote](docs/rpm/remote.md), [remote-github](docs/rpm/remote-github.md)
|
||||
- `terraform`: [remote](docs/terraform/remote.md), [local](docs/terraform/local.md)
|
||||
|
||||
### GitHub authentication
|
||||
|
||||
@@ -180,137 +104,10 @@ GITHUB_APP_PRIVATE_KEY_PATH=/etc/artifactapi/github-app.pem
|
||||
|
||||
## Terraform
|
||||
|
||||
Remotes and virtuals are managed by Terraform. Each package type has its own resource:
|
||||
|
||||
```hcl
|
||||
resource "artifactapi_remote_generic" "github" {
|
||||
name = "github"
|
||||
base_url = "https://github.com"
|
||||
|
||||
immutable_ttl = 0
|
||||
mutable_ttl = 7200
|
||||
|
||||
patterns = [
|
||||
"ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$",
|
||||
"mikefarah/yq/.*/yq_linux_amd64$",
|
||||
]
|
||||
|
||||
mutable_patterns = [
|
||||
".*/archive/refs/heads/.*\\.tar\\.gz$",
|
||||
]
|
||||
}
|
||||
|
||||
resource "artifactapi_remote_docker" "dockerhub" {
|
||||
name = "dockerhub"
|
||||
base_url = "https://registry-1.docker.io"
|
||||
|
||||
immutable_ttl = 0
|
||||
mutable_ttl = 300
|
||||
ban_tags_enabled = true
|
||||
ban_tags = ["latest"]
|
||||
|
||||
patterns = [
|
||||
"^library/postgres",
|
||||
"^library/redis",
|
||||
]
|
||||
}
|
||||
|
||||
resource "artifactapi_remote_helm" "jetstack" {
|
||||
name = "jetstack"
|
||||
base_url = "https://charts.jetstack.io"
|
||||
|
||||
immutable_ttl = 0
|
||||
mutable_ttl = 3600
|
||||
}
|
||||
|
||||
resource "artifactapi_virtual" "helm" {
|
||||
name = "helm"
|
||||
package_type = "helm"
|
||||
members = [artifactapi_remote_helm.jetstack.name]
|
||||
}
|
||||
```
|
||||
Remotes and virtuals are managed by Terraform. Each package type has its own resource; see `docs/<type>/` for examples.
|
||||
|
||||
Provider: [terraform-provider-artifactapi](../terraform-provider-artifactapi)
|
||||
|
||||
### Serving providers as a registry
|
||||
|
||||
A local `terraform` repo is a real provider registry: upload
|
||||
`terraform-provider-{type}_{version}_{os}_{arch}.zip` files under
|
||||
`{namespace}/{type}/`, and Terraform installs them from a bare source address —
|
||||
no `.terraformrc` mirror config:
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
required_providers {
|
||||
artifactapi = {
|
||||
source = "artifactapi.k8s.syd1.au.unkin.net/<repo>/<type>"
|
||||
version = "0.1.2"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The Terraform *namespace* segment is the artifactapi repo name; the provider is
|
||||
matched by *type*. The registry serves service discovery
|
||||
(`/.well-known/terraform.json`), the `providers.v1` version/download endpoints,
|
||||
and a GPG-signed `SHA256SUMS` per the provider registry protocol.
|
||||
|
||||
Signing needs a GPG key. By default artifactapi generates one on first start and
|
||||
stores it in the database (`signing_keys` table), so every replica shares it and
|
||||
there's nothing to provision. To bring your own key instead, point
|
||||
`TF_SIGNING_KEY_PATH` at an armored private key (optionally
|
||||
`TF_SIGNING_KEY_PASSPHRASE`), which takes precedence over the generated one.
|
||||
`TF_PROVIDER_PROTOCOLS` (default `5.0,6.0`) sets the advertised plugin protocols.
|
||||
|
||||
### Local helm repository
|
||||
|
||||
A local `helm` repo is a real chart repository: `PUT` a packaged chart to
|
||||
`/api/v2/remotes/<repo>/files/<chart>-<version>.tgz` and `index.yaml` is
|
||||
generated from each chart's `Chart.yaml` (with sha256 digests) at
|
||||
`/api/v1/local/<repo>/index.yaml`; a file not named after its `Chart.yaml`
|
||||
name and version is stored but left out of the index. Add the local as a
|
||||
member of a helm virtual to serve it alongside remotes.
|
||||
|
||||
```bash
|
||||
helm package ./mychart
|
||||
curl -fsSL --upload-file mychart-0.1.0.tgz \
|
||||
https://artifactapi.k8s.syd1.au.unkin.net/api/v2/remotes/<repo>/files/mychart-0.1.0.tgz
|
||||
helm repo add <repo> https://artifactapi.k8s.syd1.au.unkin.net/api/v1/local/<repo>
|
||||
```
|
||||
|
||||
### Local puppet forge
|
||||
|
||||
A local `puppet` repo is a Forge: `PUT` a module tarball to
|
||||
`/api/v2/remotes/<repo>/files/<owner>-<module>-<version>.tar.gz` and the Forge
|
||||
v3 API (`/v3/modules`, `/v3/releases`, `/v3/files`) is served from each
|
||||
release's `metadata.json` at `/api/v1/local/<repo>`. Versions are immutable.
|
||||
|
||||
```bash
|
||||
puppet module build ./hello
|
||||
curl -fsSL --upload-file pkg/unkin-hello-0.1.0.tar.gz \
|
||||
https://artifactapi.k8s.syd1.au.unkin.net/api/v2/remotes/<repo>/files/unkin-hello-0.1.0.tar.gz
|
||||
puppet module install unkin-hello \
|
||||
--module_repository https://artifactapi.k8s.syd1.au.unkin.net/api/v1/local/<repo>
|
||||
```
|
||||
|
||||
### Local docker registry
|
||||
|
||||
A local `docker` repo is a real container registry, not a mirror: it serves the
|
||||
Docker Registry HTTP API V2 for both push and pull, so any client (`docker`,
|
||||
`podman`, `skopeo`, `buildah`) can use it directly.
|
||||
|
||||
```sh
|
||||
docker tag myapp:latest artifactapi.k8s.syd1.au.unkin.net/docker-internal/myapp:latest
|
||||
docker push artifactapi.k8s.syd1.au.unkin.net/docker-internal/myapp:latest
|
||||
docker pull artifactapi.k8s.syd1.au.unkin.net/docker-internal/myapp:latest
|
||||
```
|
||||
|
||||
The first path segment after `/v2/` is the artifactapi repo name; the remainder
|
||||
is the image name. Blobs and manifests are stored through the shared
|
||||
content-addressable store (deduplicated by digest, reaped by GC once
|
||||
unreferenced); tags are mutable references and re-pushing a tag moves it. Blob
|
||||
uploads support both the monolithic and chunked (`POST`/`PATCH`/`PUT`) flows.
|
||||
|
||||
## Access Control
|
||||
|
||||
| Field | Default | Behaviour |
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
# `github_alpine` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `github_alpine` | `APKINDEX.tar.gz` (synthesized) | `.apk` (redirected) |
|
||||
@@ -0,0 +1,5 @@
|
||||
# `alpine` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `alpine` | `APKINDEX.tar.gz` | `.apk` |
|
||||
@@ -0,0 +1,5 @@
|
||||
# `cargo` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `cargo` | sparse index files (`config.json` synthesized) | `crates/*/*.crate` |
|
||||
@@ -0,0 +1,5 @@
|
||||
# `github_deb` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `github_deb` | `Packages*`, `Release*` (synthesized) | `.deb` (redirected) |
|
||||
@@ -0,0 +1,5 @@
|
||||
# `deb` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `deb` | `Packages*`, `Sources*`, `Release`, `InRelease`, `Release.gpg`, `Contents-*` | `.deb` |
|
||||
@@ -0,0 +1,17 @@
|
||||
# Local docker registry
|
||||
|
||||
A local `docker` repo is a real container registry, not a mirror: it serves the
|
||||
Docker Registry HTTP API V2 for both push and pull, so any client (`docker`,
|
||||
`podman`, `skopeo`, `buildah`) can use it directly.
|
||||
|
||||
```sh
|
||||
docker tag myapp:latest artifactapi.k8s.syd1.au.unkin.net/docker-internal/myapp:latest
|
||||
docker push artifactapi.k8s.syd1.au.unkin.net/docker-internal/myapp:latest
|
||||
docker pull artifactapi.k8s.syd1.au.unkin.net/docker-internal/myapp:latest
|
||||
```
|
||||
|
||||
The first path segment after `/v2/` is the artifactapi repo name; the remainder
|
||||
is the image name. Blobs and manifests are stored through the shared
|
||||
content-addressable store (deduplicated by digest, reaped by GC once
|
||||
unreferenced); tags are mutable references and re-pushing a tag moves it. Blob
|
||||
uploads support both the monolithic and chunked (`POST`/`PATCH`/`PUT`) flows.
|
||||
@@ -0,0 +1,24 @@
|
||||
# `docker` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `docker` | tag manifests, `/tags/list` | blobs, digest manifests |
|
||||
|
||||
## Terraform
|
||||
|
||||
```hcl
|
||||
resource "artifactapi_remote_docker" "dockerhub" {
|
||||
name = "dockerhub"
|
||||
base_url = "https://registry-1.docker.io"
|
||||
|
||||
immutable_ttl = 0
|
||||
mutable_ttl = 300
|
||||
ban_tags_enabled = true
|
||||
ban_tags = ["latest"]
|
||||
|
||||
patterns = [
|
||||
"^library/postgres",
|
||||
"^library/redis",
|
||||
]
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,26 @@
|
||||
# `generic` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `generic` | nothing | everything |
|
||||
|
||||
## Terraform
|
||||
|
||||
```hcl
|
||||
resource "artifactapi_remote_generic" "github" {
|
||||
name = "github"
|
||||
base_url = "https://github.com"
|
||||
|
||||
immutable_ttl = 0
|
||||
mutable_ttl = 7200
|
||||
|
||||
patterns = [
|
||||
"ducaale/xh/.*/xh-.*-x86_64-unknown-linux-musl.tar.gz$",
|
||||
"mikefarah/yq/.*/yq_linux_amd64$",
|
||||
]
|
||||
|
||||
mutable_patterns = [
|
||||
".*/archive/refs/heads/.*\\.tar\\.gz$",
|
||||
]
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,5 @@
|
||||
# `goproxy` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `goproxy` | `@v/list`, `@latest` | `.info`, `.mod`, `.zip` |
|
||||
@@ -0,0 +1,15 @@
|
||||
# Local helm repository
|
||||
|
||||
A local `helm` repo is a real chart repository: `PUT` a packaged chart to
|
||||
`/api/v2/remotes/<repo>/files/<chart>-<version>.tgz` and `index.yaml` is
|
||||
generated from each chart's `Chart.yaml` (with sha256 digests) at
|
||||
`/api/v1/local/<repo>/index.yaml`; a file not named after its `Chart.yaml`
|
||||
name and version is stored but left out of the index. Add the local as a
|
||||
member of a helm virtual to serve it alongside remotes.
|
||||
|
||||
```bash
|
||||
helm package ./mychart
|
||||
curl -fsSL --upload-file mychart-0.1.0.tgz \
|
||||
https://artifactapi.k8s.syd1.au.unkin.net/api/v2/remotes/<repo>/files/mychart-0.1.0.tgz
|
||||
helm repo add <repo> https://artifactapi.k8s.syd1.au.unkin.net/api/v1/local/<repo>
|
||||
```
|
||||
@@ -0,0 +1,17 @@
|
||||
# `helm` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `helm` | `index.yaml` | `.tgz` charts |
|
||||
|
||||
## Terraform
|
||||
|
||||
```hcl
|
||||
resource "artifactapi_remote_helm" "jetstack" {
|
||||
name = "jetstack"
|
||||
base_url = "https://charts.jetstack.io"
|
||||
|
||||
immutable_ttl = 0
|
||||
mutable_ttl = 3600
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,9 @@
|
||||
# `helm` virtual
|
||||
|
||||
```hcl
|
||||
resource "artifactapi_virtual" "helm" {
|
||||
name = "helm"
|
||||
package_type = "helm"
|
||||
members = [artifactapi_remote_helm.jetstack.name]
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,5 @@
|
||||
# `npm` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `npm` | package metadata | `.tgz` tarballs |
|
||||
@@ -0,0 +1,5 @@
|
||||
# `nuget` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `nuget` | `index.json` (service index), registration and flat container indexes | `.nupkg`, `.nuspec` |
|
||||
@@ -0,0 +1,14 @@
|
||||
# Local puppet forge
|
||||
|
||||
A local `puppet` repo is a Forge: `PUT` a module tarball to
|
||||
`/api/v2/remotes/<repo>/files/<owner>-<module>-<version>.tar.gz` and the Forge
|
||||
v3 API (`/v3/modules`, `/v3/releases`, `/v3/files`) is served from each
|
||||
release's `metadata.json` at `/api/v1/local/<repo>`. Versions are immutable.
|
||||
|
||||
```bash
|
||||
puppet module build ./hello
|
||||
curl -fsSL --upload-file pkg/unkin-hello-0.1.0.tar.gz \
|
||||
https://artifactapi.k8s.syd1.au.unkin.net/api/v2/remotes/<repo>/files/unkin-hello-0.1.0.tar.gz
|
||||
puppet module install unkin-hello \
|
||||
--module_repository https://artifactapi.k8s.syd1.au.unkin.net/api/v1/local/<repo>
|
||||
```
|
||||
@@ -0,0 +1,5 @@
|
||||
# `puppet` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `puppet` | `v3/modules/*`, `v3/releases*` | `.tar.gz` |
|
||||
@@ -0,0 +1,5 @@
|
||||
# `pypi` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `pypi` | `simple/*` index pages | `.whl`, `.tar.gz` |
|
||||
@@ -0,0 +1,80 @@
|
||||
# `github_rpm` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `github_rpm` | `repodata/*` (synthesized) | `.rpm` (redirected) |
|
||||
|
||||
## GitHub releases as a yum repo (metadata-only, no precache)
|
||||
|
||||
A `github_rpm` remote turns a GitHub repo's **releases** into a real `dnf`/`yum`
|
||||
repository without ever caching the packages. It scans releases for `.rpm`
|
||||
assets, derives each package's metadata (NEVRA, requires/provides/conflicts/
|
||||
obsoletes, files, checksum) and **synthesizes `repodata/` on the fly**. Package
|
||||
metadata comes from a **ranged GET of just the RPM header** (the header sits at
|
||||
the front of the file, so the whole package is never downloaded); the sha256
|
||||
checksum comes from the GitHub asset `digest` when present, else a one-time
|
||||
lazy stream. Derived metadata is cached (keyed by asset) so repodata generation
|
||||
is served from primed DB rows, never a cold on-demand derive.
|
||||
|
||||
Each package's `<location>` points back at the remote, which **302-redirects**
|
||||
the download to the `releases_remote` — an existing generic `github.com` remote
|
||||
that streams the actual bytes. `dnf` follows the redirect transparently.
|
||||
|
||||
### Background syncer
|
||||
|
||||
A single process-wide **background syncer** keeps every `github_rpm` remote's
|
||||
derived metadata current off the client request path:
|
||||
|
||||
- **Prime on create.** Creating a `github_rpm` remote enqueues a background prime
|
||||
scan, so its metadata is derived right away without blocking the create call.
|
||||
The first `dnf` request is served from cache. If a request arrives before the
|
||||
prime lands, it returns a retryable `503` (with `Retry-After`) rather than
|
||||
serving an empty repo or blocking on a multi-minute derive.
|
||||
- **Periodic re-check, driven by `mutable_ttl`.** Each remote is re-checked for
|
||||
new or changed releases no more often than its `mutable_ttl`. New/changed
|
||||
assets are derived incrementally; assets already cached are never re-fetched,
|
||||
and assets that disappear upstream are pruned.
|
||||
- **ETag / 304 conditional requests.** The releases-list `ETag` is stored per
|
||||
remote and sent as `If-None-Match`; a `304 Not Modified` means nothing changed
|
||||
and the syncer derives nothing. GitHub does not count `304` conditional
|
||||
responses against the rate limit, so an unchanged repo is nearly free — this is
|
||||
the main lever keeping GitHub traffic low.
|
||||
- **Global rate limit.** Every GitHub call (releases list + each ranged asset
|
||||
header GET) passes through a single token-bucket limiter **shared across all
|
||||
remotes**, so GitHub is never hammered. Configure a token (`password`) on the
|
||||
remote for the higher authenticated rate limit (~5000/hr vs ~60/hr
|
||||
unauthenticated).
|
||||
- **Multi-replica coordination.** State is shared through the database. Before a
|
||||
periodic scan a replica must atomically claim a per-remote lease
|
||||
(`github_rpm_sync_state`: `last_synced_at`, `etag`, `sync_lease_owner`,
|
||||
`sync_lease_expires`); only the winner scans. This bounds total GitHub load to
|
||||
~once per `mutable_ttl` regardless of replica count, and the shared `etag`
|
||||
lets any replica issue the conditional request.
|
||||
|
||||
```hcl
|
||||
# Backend that serves the actual .rpm bytes from github.com.
|
||||
resource "artifactapi_remote_generic" "github" {
|
||||
name = "github"
|
||||
base_url = "https://github.com"
|
||||
patterns = [
|
||||
"acme/tools/releases/download/.*\\.rpm$", # allowlist the repo's release assets
|
||||
]
|
||||
}
|
||||
|
||||
resource "artifactapi_remote_github_rpm" "acme-tools" {
|
||||
name = "acme-tools"
|
||||
base_url = "https://api.github.com/repos/acme/tools" # the releases API root
|
||||
releases_remote = "github" # backend for downloads
|
||||
mutable_ttl = 3600 # release re-scan interval
|
||||
|
||||
# Optional: restrict which release assets become packages (regex on filename).
|
||||
patterns = [".*\\.x86_64\\.rpm$", ".*\\.noarch\\.rpm$"]
|
||||
|
||||
# Optional: a token for private repos / higher API rate limits.
|
||||
# password = "ghp_..."
|
||||
}
|
||||
```
|
||||
|
||||
`dnf` config: `baseurl=https://artifactapi.example/api/v1/remote/acme-tools`.
|
||||
The repo is multi-arch (no `$basearch` needed) — `dnf` selects matching packages
|
||||
from the synthesized metadata.
|
||||
@@ -0,0 +1,5 @@
|
||||
# `rpm` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `rpm` | `repomd.xml`, `repodata/*` | `.rpm` |
|
||||
@@ -0,0 +1,29 @@
|
||||
# Serving providers as a registry
|
||||
|
||||
A local `terraform` repo is a real provider registry: upload
|
||||
`terraform-provider-{type}_{version}_{os}_{arch}.zip` files under
|
||||
`{namespace}/{type}/`, and Terraform installs them from a bare source address —
|
||||
no `.terraformrc` mirror config:
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
required_providers {
|
||||
artifactapi = {
|
||||
source = "artifactapi.k8s.syd1.au.unkin.net/<repo>/<type>"
|
||||
version = "0.1.2"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The Terraform *namespace* segment is the artifactapi repo name; the provider is
|
||||
matched by *type*. The registry serves service discovery
|
||||
(`/.well-known/terraform.json`), the `providers.v1` version/download endpoints,
|
||||
and a GPG-signed `SHA256SUMS` per the provider registry protocol.
|
||||
|
||||
Signing needs a GPG key. By default artifactapi generates one on first start and
|
||||
stores it in the database (`signing_keys` table), so every replica shares it and
|
||||
there's nothing to provision. To bring your own key instead, point
|
||||
`TF_SIGNING_KEY_PATH` at an armored private key (optionally
|
||||
`TF_SIGNING_KEY_PASSPHRASE`), which takes precedence over the generated one.
|
||||
`TF_PROVIDER_PROTOCOLS` (default `5.0,6.0`) sets the advertised plugin protocols.
|
||||
@@ -0,0 +1,5 @@
|
||||
# `terraform` remote
|
||||
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `terraform` | `*/versions` | `*/download/*/*` |
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"alpine", "alpine/x86_64/testpkg-1.0-r0.apk", "alpine/x86_64/testpkg-1.0-r0.apk"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"cargo", "crates/mycrate/mycrate-1.0.0.crate", "crates/mycrate/mycrate-1.0.0.crate"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"docker", "library/testimg/blobs/blobdata", "v2/library/testimg/blobs/blobdata"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"generic", "blobs/hello.bin", "blobs/hello.bin"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"goproxy", "goproxy/example.com/mod/@v/v1.0.0.zip", "goproxy/example.com/mod/@v/v1.0.0.zip"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"helm", "charts/mychart-1.0.0.tgz", "charts/mychart-1.0.0.tgz"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"npm", "mypkg/-/mypkg-1.0.0.tgz", "mypkg/-/mypkg-1.0.0.tgz"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"nuget", "nuget/e2epkg/1.0.0/e2epkg.1.0.0.nupkg", "nuget/e2epkg/1.0.0/e2epkg.1.0.0.nupkg"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"puppet", "puppet-releases/author-mod-1.0.0.tar.gz", "puppet-releases/author-mod-1.0.0.tar.gz"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"pypi", "packages/foo-1.0-py3-none-any.whl", "packages/foo-1.0-py3-none-any.whl"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"rpm", "rpmrepo/Packages/e2e-testpkg-1.0-1.noarch.rpm", "rpmrepo/Packages/e2e-testpkg-1.0-1.noarch.rpm"})
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
//go:build dockere2e
|
||||
|
||||
package e2edocker
|
||||
|
||||
func init() {
|
||||
cachingCases = append(cachingCases, cachingCase{"terraform", "hashicorp/aws/download/pkg.zip", "v1/providers/hashicorp/aws/download/pkg.zip"})
|
||||
}
|
||||
+13
-23
@@ -9,33 +9,23 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
type cachingCase struct {
|
||||
pkgType string
|
||||
// path is the request path under /api/v1/remote/<name>/. The provider
|
||||
// derives the upstream URL from it (docker prepends /v2/, terraform
|
||||
// prepends /v1/providers/), and the fixture lives at that resolved path.
|
||||
path string
|
||||
fixture string
|
||||
}
|
||||
|
||||
// cachingCases is filled by one caching_<type>_test.go file per package type.
|
||||
var cachingCases []cachingCase
|
||||
|
||||
// TestCachingPerProvider proxies one immutable artifact for every remote
|
||||
// package type through the mock upstream and asserts: first fetch is served
|
||||
// from the remote, the second from cache, and the bytes match the origin.
|
||||
func TestCachingPerProvider(t *testing.T) {
|
||||
cases := []struct {
|
||||
pkgType string
|
||||
// path is the request path under /api/v1/remote/<name>/. The provider
|
||||
// derives the upstream URL from it (docker prepends /v2/, terraform
|
||||
// prepends /v1/providers/), and the fixture lives at that resolved path.
|
||||
path string
|
||||
fixture string
|
||||
}{
|
||||
{"generic", "blobs/hello.bin", "blobs/hello.bin"},
|
||||
{"npm", "mypkg/-/mypkg-1.0.0.tgz", "mypkg/-/mypkg-1.0.0.tgz"},
|
||||
{"helm", "charts/mychart-1.0.0.tgz", "charts/mychart-1.0.0.tgz"},
|
||||
{"pypi", "packages/foo-1.0-py3-none-any.whl", "packages/foo-1.0-py3-none-any.whl"},
|
||||
{"rpm", "rpmrepo/Packages/e2e-testpkg-1.0-1.noarch.rpm", "rpmrepo/Packages/e2e-testpkg-1.0-1.noarch.rpm"},
|
||||
{"alpine", "alpine/x86_64/testpkg-1.0-r0.apk", "alpine/x86_64/testpkg-1.0-r0.apk"},
|
||||
{"puppet", "puppet-releases/author-mod-1.0.0.tar.gz", "puppet-releases/author-mod-1.0.0.tar.gz"},
|
||||
{"goproxy", "goproxy/example.com/mod/@v/v1.0.0.zip", "goproxy/example.com/mod/@v/v1.0.0.zip"},
|
||||
{"cargo", "crates/mycrate/mycrate-1.0.0.crate", "crates/mycrate/mycrate-1.0.0.crate"},
|
||||
{"nuget", "nuget/e2epkg/1.0.0/e2epkg.1.0.0.nupkg", "nuget/e2epkg/1.0.0/e2epkg.1.0.0.nupkg"},
|
||||
{"terraform", "hashicorp/aws/download/pkg.zip", "v1/providers/hashicorp/aws/download/pkg.zip"},
|
||||
{"docker", "library/testimg/blobs/blobdata", "v2/library/testimg/blobs/blobdata"},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
for _, tc := range cachingCases {
|
||||
t.Run(tc.pkgType, func(t *testing.T) {
|
||||
name := "cache-" + tc.pkgType
|
||||
createRepo(t, fmt.Sprintf(`{
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/alpine"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/cargo"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/deb"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/docker"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/generic"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/goproxy"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/helm"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/npm"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/nuget"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/puppet"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/pypi"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/rpm"
|
||||
@@ -0,0 +1,3 @@
|
||||
package all
|
||||
|
||||
import _ "git.unkin.net/unkin/artifactapi/internal/provider/terraform"
|
||||
@@ -0,0 +1,15 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
)
|
||||
|
||||
type FileStore interface {
|
||||
ListFilesByPrefix(ctx context.Context, repoName, prefix string) ([]FileEntry, error)
|
||||
ListPackages(ctx context.Context, repoName string) ([]string, error)
|
||||
}
|
||||
|
||||
type BlobReader interface {
|
||||
Download(ctx context.Context, key string) (io.ReadCloser, int64, error)
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
package provider
|
||||
|
||||
type IndexMerger interface {
|
||||
MergeIndexes(members []MemberIndex, proxyBaseURL string) ([]byte, error)
|
||||
}
|
||||
|
||||
type MemberIndex struct {
|
||||
RemoteName string
|
||||
Body []byte
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
type LocalIndexer interface {
|
||||
ServeLocalIndex(w http.ResponseWriter, r *http.Request, files FileStore, repoName, path string) bool
|
||||
GenerateLocalIndex(ctx context.Context, files FileStore, repoName, path string) ([]byte, error)
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package provider
|
||||
|
||||
type LocalUploader interface {
|
||||
ValidateUpload(filePath string) (storagePath, contentType string, err error)
|
||||
UploadResponse(storagePath, contentHash string, sizeBytes int64) map[string]any
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
)
|
||||
|
||||
// PostDeleteHook lets a provider clean up derived state (e.g. RPM metadata that
|
||||
// feeds generated repodata) after a local file is removed.
|
||||
type PostDeleteHook interface {
|
||||
AfterDelete(ctx context.Context, repoName, storagePath string, db MetadataDeleter) error
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
)
|
||||
|
||||
type PostUploadHook interface {
|
||||
AfterUpload(ctx context.Context, repoName, storagePath, contentHash string, blobs BlobReader, db MetadataStore)
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"git.unkin.net/unkin/artifactapi/pkg/models"
|
||||
)
|
||||
|
||||
// RemoteServer lets a remote provider fully answer a request itself instead of
|
||||
// going through the byte-proxy engine. It is the remote-side analog of
|
||||
// LocalIndexer: a metadata-only remote (e.g. github_rpm) uses it to synthesize
|
||||
// repodata from derived per-asset metadata and to redirect package downloads to
|
||||
// a backend remote, without ever precaching the packages. Returning false lets
|
||||
// the normal proxy path take over.
|
||||
type RemoteServer interface {
|
||||
ServeRemote(w http.ResponseWriter, r *http.Request, remote models.Remote, path, proxyBaseURL string, store RemoteMetadataStore) bool
|
||||
}
|
||||
|
||||
// RemoteMetadataStore is the persistence surface a RemoteServer needs to cache
|
||||
// and read the metadata it derives per upstream asset. *database.DB satisfies it.
|
||||
type RemoteMetadataStore interface {
|
||||
RPMMetadataReader
|
||||
MetadataStore
|
||||
MetadataDeleter
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"git.unkin.net/unkin/artifactapi/pkg/models"
|
||||
)
|
||||
|
||||
// ServeRewriter rewrites a cached mutable remote response per request, for
|
||||
// indexes whose URLs must name the requesting host and so cannot be rewritten
|
||||
// before caching.
|
||||
type ServeRewriter interface {
|
||||
RewriteServed(body []byte, remote models.Remote, proxyBaseURL string) []byte
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
)
|
||||
|
||||
// AlpineMetadataStore / AlpineMetadataDeleter / AlpineMetadataReader are the
|
||||
// Alpine-specific persistence surfaces. They are kept separate from the shared
|
||||
// RPM/Deb metadata interfaces so the apk provider can type-assert the generic
|
||||
// MetadataStore/MetadataDeleter/FileStore it is handed without widening (and
|
||||
// thus perturbing the test doubles of) the rpm and deb providers. *database.DB
|
||||
// satisfies all three.
|
||||
type AlpineMetadataStore interface {
|
||||
InsertAlpineMetadata(ctx context.Context, meta *AlpineMetadata) error
|
||||
}
|
||||
|
||||
type AlpineMetadataDeleter interface {
|
||||
DeleteAlpineMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type AlpineMetadataReader interface {
|
||||
ListAlpineMetadataEntries(ctx context.Context, repoName string) ([]AlpineMetadata, error)
|
||||
}
|
||||
|
||||
// AlpineMetadata is the derived per-package metadata for an Alpine .apk, holding
|
||||
// the fields an APKINDEX record carries plus the apk pull checksum (Q1…, the
|
||||
// sha1 of the control gzip stream) and the download/installed sizes.
|
||||
type AlpineMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Checksum string // C: "Q1" + base64(sha1(control gzip stream))
|
||||
Name string // P:
|
||||
Version string // V:
|
||||
Arch string // A:
|
||||
DownloadSize int64 // S: on-disk .apk size
|
||||
InstalledSize int64 // I: unpacked size from .PKGINFO
|
||||
Description string // T:
|
||||
URL string // U:
|
||||
License string // L:
|
||||
Origin string // o:
|
||||
Maintainer string // m:
|
||||
BuildTime int64 // t:
|
||||
Commit string // c:
|
||||
ProviderPriority string // k:
|
||||
Depends []string // D:
|
||||
Provides []string // p:
|
||||
InstallIf []string // i:
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DebMetadataReader is the read surface the deb LocalIndexer needs to
|
||||
// regenerate a flat apt repository (Packages/Release) from stored rows.
|
||||
// *database.DB satisfies it; ServeLocalIndex type-asserts the FileStore to it,
|
||||
// mirroring how the rpm provider reaches its RPMMetadataReader.
|
||||
type DebMetadataReader interface {
|
||||
ListDebMetadataEntries(ctx context.Context, repoName string) ([]DebMetadata, error)
|
||||
}
|
||||
|
||||
// DebMetadata is the derived per-package metadata for a Debian .deb, carrying
|
||||
// the full raw control stanza so the Packages index can be regenerated
|
||||
// faithfully alongside the computed size/md5/sha256 apt requires.
|
||||
type DebMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Name string
|
||||
Version string
|
||||
Architecture string
|
||||
Control string
|
||||
Size int64
|
||||
MD5 string
|
||||
SHA256 string
|
||||
// CreatedAt is the persisted insert time; the Release Date: is derived from
|
||||
// the newest value so the index is byte-identical across replicas and
|
||||
// regenerations (issue #117) rather than stamped from wall clock.
|
||||
CreatedAt time.Time
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
type HelmMetadataStore interface {
|
||||
InsertHelmMetadata(ctx context.Context, meta *HelmMetadata) error
|
||||
}
|
||||
|
||||
type HelmMetadataDeleter interface {
|
||||
DeleteHelmMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type HelmMetadataReader interface {
|
||||
ListHelmMetadataEntries(ctx context.Context, repoName string) ([]HelmMetadata, error)
|
||||
}
|
||||
|
||||
// HelmMetadata is a local chart's Chart.yaml, kept verbatim so index.yaml
|
||||
// entries carry every field helm reads.
|
||||
type HelmMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Name string
|
||||
Version string
|
||||
ChartYAML string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
type PuppetMetadataStore interface {
|
||||
InsertPuppetMetadata(ctx context.Context, meta *PuppetMetadata) error
|
||||
}
|
||||
|
||||
type PuppetMetadataDeleter interface {
|
||||
DeletePuppetMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type PuppetMetadataReader interface {
|
||||
ListPuppetMetadataEntries(ctx context.Context, repoName, slug string) ([]PuppetMetadata, error)
|
||||
}
|
||||
|
||||
// PuppetMetadata is a local module release; MetadataJSON is the tarball's
|
||||
// metadata.json verbatim so Forge release objects carry every field.
|
||||
type PuppetMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Slug string
|
||||
Version string
|
||||
MetadataJSON string
|
||||
Size int64
|
||||
MD5 string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
type RPMMetadataReader interface {
|
||||
ListRPMMetadataEntries(ctx context.Context, repoName string) ([]RPMMetadata, error)
|
||||
}
|
||||
|
||||
type RPMMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Name string
|
||||
Epoch int
|
||||
Version string
|
||||
Release string
|
||||
Arch string
|
||||
Summary string
|
||||
Description string
|
||||
RPMSize int64
|
||||
InstalledSize int64
|
||||
License string
|
||||
Vendor string
|
||||
Group string
|
||||
BuildHost string
|
||||
SourceRPM string
|
||||
URL string
|
||||
Packager string
|
||||
Requires []RPMDep
|
||||
Provides []RPMDep
|
||||
Conflicts []RPMDep
|
||||
Obsoletes []RPMDep
|
||||
Files []RPMFile
|
||||
Changelogs []RPMChangelog
|
||||
// CreatedAt is the persisted upload timestamp; used as a stable, replica-independent
|
||||
// value for the repodata <time>/<revision> fields so generated indexes are deterministic.
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
type RPMDep struct {
|
||||
Name string `json:"name"`
|
||||
Flags string `json:"flags,omitempty"`
|
||||
Epoch string `json:"epoch,omitempty"`
|
||||
Version string `json:"version,omitempty"`
|
||||
Release string `json:"release,omitempty"`
|
||||
}
|
||||
|
||||
type RPMFile struct {
|
||||
Path string `json:"path"`
|
||||
Type string `json:"type,omitempty"`
|
||||
}
|
||||
|
||||
type RPMChangelog struct {
|
||||
Author string `json:"author"`
|
||||
Date int64 `json:"date"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
)
|
||||
|
||||
type MetadataStore interface {
|
||||
InsertRPMMetadata(ctx context.Context, meta *RPMMetadata) error
|
||||
InsertDebMetadata(ctx context.Context, meta *DebMetadata) error
|
||||
}
|
||||
|
||||
type MetadataDeleter interface {
|
||||
DeleteRPMMetadata(ctx context.Context, repoName, filePath string) error
|
||||
DeleteDebMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
@@ -3,9 +3,7 @@ package provider
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"git.unkin.net/unkin/artifactapi/pkg/models"
|
||||
)
|
||||
@@ -31,205 +29,6 @@ type FileEntry struct {
|
||||
ContentHash string
|
||||
}
|
||||
|
||||
type FileStore interface {
|
||||
ListFilesByPrefix(ctx context.Context, repoName, prefix string) ([]FileEntry, error)
|
||||
ListPackages(ctx context.Context, repoName string) ([]string, error)
|
||||
}
|
||||
|
||||
type LocalUploader interface {
|
||||
ValidateUpload(filePath string) (storagePath, contentType string, err error)
|
||||
UploadResponse(storagePath, contentHash string, sizeBytes int64) map[string]any
|
||||
}
|
||||
|
||||
type LocalIndexer interface {
|
||||
ServeLocalIndex(w http.ResponseWriter, r *http.Request, files FileStore, repoName, path string) bool
|
||||
GenerateLocalIndex(ctx context.Context, files FileStore, repoName, path string) ([]byte, error)
|
||||
}
|
||||
|
||||
type BlobReader interface {
|
||||
Download(ctx context.Context, key string) (io.ReadCloser, int64, error)
|
||||
}
|
||||
|
||||
type PostUploadHook interface {
|
||||
AfterUpload(ctx context.Context, repoName, storagePath, contentHash string, blobs BlobReader, db MetadataStore)
|
||||
}
|
||||
|
||||
// PostDeleteHook lets a provider clean up derived state (e.g. RPM metadata that
|
||||
// feeds generated repodata) after a local file is removed.
|
||||
type PostDeleteHook interface {
|
||||
AfterDelete(ctx context.Context, repoName, storagePath string, db MetadataDeleter) error
|
||||
}
|
||||
|
||||
type MetadataStore interface {
|
||||
InsertRPMMetadata(ctx context.Context, meta *RPMMetadata) error
|
||||
InsertDebMetadata(ctx context.Context, meta *DebMetadata) error
|
||||
}
|
||||
|
||||
// RemoteServer lets a remote provider fully answer a request itself instead of
|
||||
// going through the byte-proxy engine. It is the remote-side analog of
|
||||
// LocalIndexer: a metadata-only remote (e.g. github_rpm) uses it to synthesize
|
||||
// repodata from derived per-asset metadata and to redirect package downloads to
|
||||
// a backend remote, without ever precaching the packages. Returning false lets
|
||||
// the normal proxy path take over.
|
||||
type RemoteServer interface {
|
||||
ServeRemote(w http.ResponseWriter, r *http.Request, remote models.Remote, path, proxyBaseURL string, store RemoteMetadataStore) bool
|
||||
}
|
||||
|
||||
// ServeRewriter rewrites a cached mutable remote response per request, for
|
||||
// indexes whose URLs must name the requesting host and so cannot be rewritten
|
||||
// before caching.
|
||||
type ServeRewriter interface {
|
||||
RewriteServed(body []byte, remote models.Remote, proxyBaseURL string) []byte
|
||||
}
|
||||
|
||||
// RemoteMetadataStore is the persistence surface a RemoteServer needs to cache
|
||||
// and read the metadata it derives per upstream asset. *database.DB satisfies it.
|
||||
type RemoteMetadataStore interface {
|
||||
RPMMetadataReader
|
||||
MetadataStore
|
||||
MetadataDeleter
|
||||
}
|
||||
|
||||
type MetadataDeleter interface {
|
||||
DeleteRPMMetadata(ctx context.Context, repoName, filePath string) error
|
||||
DeleteDebMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type RPMMetadataReader interface {
|
||||
ListRPMMetadataEntries(ctx context.Context, repoName string) ([]RPMMetadata, error)
|
||||
}
|
||||
|
||||
// DebMetadataReader is the read surface the deb LocalIndexer needs to
|
||||
// regenerate a flat apt repository (Packages/Release) from stored rows.
|
||||
// *database.DB satisfies it; ServeLocalIndex type-asserts the FileStore to it,
|
||||
// mirroring how the rpm provider reaches its RPMMetadataReader.
|
||||
type DebMetadataReader interface {
|
||||
ListDebMetadataEntries(ctx context.Context, repoName string) ([]DebMetadata, error)
|
||||
}
|
||||
|
||||
// DebMetadata is the derived per-package metadata for a Debian .deb, carrying
|
||||
// the full raw control stanza so the Packages index can be regenerated
|
||||
// faithfully alongside the computed size/md5/sha256 apt requires.
|
||||
type DebMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Name string
|
||||
Version string
|
||||
Architecture string
|
||||
Control string
|
||||
Size int64
|
||||
MD5 string
|
||||
SHA256 string
|
||||
// CreatedAt is the persisted insert time; the Release Date: is derived from
|
||||
// the newest value so the index is byte-identical across replicas and
|
||||
// regenerations (issue #117) rather than stamped from wall clock.
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
// AlpineMetadataStore / AlpineMetadataDeleter / AlpineMetadataReader are the
|
||||
// Alpine-specific persistence surfaces. They are kept separate from the shared
|
||||
// RPM/Deb metadata interfaces so the apk provider can type-assert the generic
|
||||
// MetadataStore/MetadataDeleter/FileStore it is handed without widening (and
|
||||
// thus perturbing the test doubles of) the rpm and deb providers. *database.DB
|
||||
// satisfies all three.
|
||||
type AlpineMetadataStore interface {
|
||||
InsertAlpineMetadata(ctx context.Context, meta *AlpineMetadata) error
|
||||
}
|
||||
|
||||
type AlpineMetadataDeleter interface {
|
||||
DeleteAlpineMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type AlpineMetadataReader interface {
|
||||
ListAlpineMetadataEntries(ctx context.Context, repoName string) ([]AlpineMetadata, error)
|
||||
}
|
||||
|
||||
// AlpineMetadata is the derived per-package metadata for an Alpine .apk, holding
|
||||
// the fields an APKINDEX record carries plus the apk pull checksum (Q1…, the
|
||||
// sha1 of the control gzip stream) and the download/installed sizes.
|
||||
type AlpineMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Checksum string // C: "Q1" + base64(sha1(control gzip stream))
|
||||
Name string // P:
|
||||
Version string // V:
|
||||
Arch string // A:
|
||||
DownloadSize int64 // S: on-disk .apk size
|
||||
InstalledSize int64 // I: unpacked size from .PKGINFO
|
||||
Description string // T:
|
||||
URL string // U:
|
||||
License string // L:
|
||||
Origin string // o:
|
||||
Maintainer string // m:
|
||||
BuildTime int64 // t:
|
||||
Commit string // c:
|
||||
ProviderPriority string // k:
|
||||
Depends []string // D:
|
||||
Provides []string // p:
|
||||
InstallIf []string // i:
|
||||
}
|
||||
|
||||
type RPMMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Name string
|
||||
Epoch int
|
||||
Version string
|
||||
Release string
|
||||
Arch string
|
||||
Summary string
|
||||
Description string
|
||||
RPMSize int64
|
||||
InstalledSize int64
|
||||
License string
|
||||
Vendor string
|
||||
Group string
|
||||
BuildHost string
|
||||
SourceRPM string
|
||||
URL string
|
||||
Packager string
|
||||
Requires []RPMDep
|
||||
Provides []RPMDep
|
||||
Conflicts []RPMDep
|
||||
Obsoletes []RPMDep
|
||||
Files []RPMFile
|
||||
Changelogs []RPMChangelog
|
||||
// CreatedAt is the persisted upload timestamp; used as a stable, replica-independent
|
||||
// value for the repodata <time>/<revision> fields so generated indexes are deterministic.
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
type RPMDep struct {
|
||||
Name string `json:"name"`
|
||||
Flags string `json:"flags,omitempty"`
|
||||
Epoch string `json:"epoch,omitempty"`
|
||||
Version string `json:"version,omitempty"`
|
||||
Release string `json:"release,omitempty"`
|
||||
}
|
||||
|
||||
type RPMFile struct {
|
||||
Path string `json:"path"`
|
||||
Type string `json:"type,omitempty"`
|
||||
}
|
||||
|
||||
type RPMChangelog struct {
|
||||
Author string `json:"author"`
|
||||
Date int64 `json:"date"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
type IndexMerger interface {
|
||||
MergeIndexes(members []MemberIndex, proxyBaseURL string) ([]byte, error)
|
||||
}
|
||||
|
||||
type MemberIndex struct {
|
||||
RemoteName string
|
||||
Body []byte
|
||||
}
|
||||
|
||||
var registry = map[models.PackageType]Provider{}
|
||||
|
||||
func Register(p Provider) {
|
||||
@@ -247,53 +46,3 @@ func Get(t models.PackageType) (Provider, error) {
|
||||
func All() map[models.PackageType]Provider {
|
||||
return registry
|
||||
}
|
||||
|
||||
type HelmMetadataStore interface {
|
||||
InsertHelmMetadata(ctx context.Context, meta *HelmMetadata) error
|
||||
}
|
||||
|
||||
type HelmMetadataDeleter interface {
|
||||
DeleteHelmMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type HelmMetadataReader interface {
|
||||
ListHelmMetadataEntries(ctx context.Context, repoName string) ([]HelmMetadata, error)
|
||||
}
|
||||
|
||||
// HelmMetadata is a local chart's Chart.yaml, kept verbatim so index.yaml
|
||||
// entries carry every field helm reads.
|
||||
type HelmMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Name string
|
||||
Version string
|
||||
ChartYAML string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
type PuppetMetadataStore interface {
|
||||
InsertPuppetMetadata(ctx context.Context, meta *PuppetMetadata) error
|
||||
}
|
||||
|
||||
type PuppetMetadataDeleter interface {
|
||||
DeletePuppetMetadata(ctx context.Context, repoName, filePath string) error
|
||||
}
|
||||
|
||||
type PuppetMetadataReader interface {
|
||||
ListPuppetMetadataEntries(ctx context.Context, repoName, slug string) ([]PuppetMetadata, error)
|
||||
}
|
||||
|
||||
// PuppetMetadata is a local module release; MetadataJSON is the tarball's
|
||||
// metadata.json verbatim so Forge release objects carry every field.
|
||||
type PuppetMetadata struct {
|
||||
RepoName string
|
||||
FilePath string
|
||||
ContentHash string
|
||||
Slug string
|
||||
Version string
|
||||
MetadataJSON string
|
||||
Size int64
|
||||
MD5 string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
@@ -21,19 +21,10 @@ import (
|
||||
"git.unkin.net/unkin/artifactapi/internal/database"
|
||||
"git.unkin.net/unkin/artifactapi/internal/gc"
|
||||
"git.unkin.net/unkin/artifactapi/internal/githubauth"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/all"
|
||||
"git.unkin.net/unkin/artifactapi/internal/provider/alpine"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/cargo"
|
||||
"git.unkin.net/unkin/artifactapi/internal/provider/deb"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/docker"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/generic"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/goproxy"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/helm"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/npm"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/nuget"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/puppet"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/pypi"
|
||||
"git.unkin.net/unkin/artifactapi/internal/provider/rpm"
|
||||
_ "git.unkin.net/unkin/artifactapi/internal/provider/terraform"
|
||||
"git.unkin.net/unkin/artifactapi/internal/proxy"
|
||||
"git.unkin.net/unkin/artifactapi/internal/storage"
|
||||
"git.unkin.net/unkin/artifactapi/internal/tfsign"
|
||||
|
||||
@@ -4,42 +4,10 @@ import "fmt"
|
||||
|
||||
type PackageType string
|
||||
|
||||
const (
|
||||
PackageGeneric PackageType = "generic"
|
||||
PackageDocker PackageType = "docker"
|
||||
PackageHelm PackageType = "helm"
|
||||
PackagePyPI PackageType = "pypi"
|
||||
PackageNPM PackageType = "npm"
|
||||
PackageRPM PackageType = "rpm"
|
||||
PackageDeb PackageType = "deb"
|
||||
PackageAlpine PackageType = "alpine"
|
||||
PackagePuppet PackageType = "puppet"
|
||||
PackageTerraform PackageType = "terraform"
|
||||
PackageGoProxy PackageType = "goproxy"
|
||||
PackageCargo PackageType = "cargo"
|
||||
PackageNuGet PackageType = "nuget"
|
||||
PackageGitHubRPM PackageType = "github_rpm"
|
||||
PackageGitHubDeb PackageType = "github_deb"
|
||||
PackageGitHubAlpine PackageType = "github_alpine"
|
||||
)
|
||||
var validPackageTypes = map[PackageType]bool{}
|
||||
|
||||
var validPackageTypes = map[PackageType]bool{
|
||||
PackageGeneric: true,
|
||||
PackageDocker: true,
|
||||
PackageHelm: true,
|
||||
PackagePyPI: true,
|
||||
PackageNPM: true,
|
||||
PackageRPM: true,
|
||||
PackageDeb: true,
|
||||
PackageAlpine: true,
|
||||
PackagePuppet: true,
|
||||
PackageTerraform: true,
|
||||
PackageGoProxy: true,
|
||||
PackageCargo: true,
|
||||
PackageNuGet: true,
|
||||
PackageGitHubRPM: true,
|
||||
PackageGitHubDeb: true,
|
||||
PackageGitHubAlpine: true,
|
||||
func registerPackageType(p PackageType) {
|
||||
validPackageTypes[p] = true
|
||||
}
|
||||
|
||||
func (p PackageType) Valid() bool {
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageAlpine PackageType = "alpine"
|
||||
|
||||
func init() { registerPackageType(PackageAlpine) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageCargo PackageType = "cargo"
|
||||
|
||||
func init() { registerPackageType(PackageCargo) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageDeb PackageType = "deb"
|
||||
|
||||
func init() { registerPackageType(PackageDeb) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageDocker PackageType = "docker"
|
||||
|
||||
func init() { registerPackageType(PackageDocker) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageGeneric PackageType = "generic"
|
||||
|
||||
func init() { registerPackageType(PackageGeneric) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageGitHubAlpine PackageType = "github_alpine"
|
||||
|
||||
func init() { registerPackageType(PackageGitHubAlpine) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageGitHubDeb PackageType = "github_deb"
|
||||
|
||||
func init() { registerPackageType(PackageGitHubDeb) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageGitHubRPM PackageType = "github_rpm"
|
||||
|
||||
func init() { registerPackageType(PackageGitHubRPM) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageGoProxy PackageType = "goproxy"
|
||||
|
||||
func init() { registerPackageType(PackageGoProxy) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageHelm PackageType = "helm"
|
||||
|
||||
func init() { registerPackageType(PackageHelm) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageNPM PackageType = "npm"
|
||||
|
||||
func init() { registerPackageType(PackageNPM) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageNuGet PackageType = "nuget"
|
||||
|
||||
func init() { registerPackageType(PackageNuGet) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackagePuppet PackageType = "puppet"
|
||||
|
||||
func init() { registerPackageType(PackagePuppet) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackagePyPI PackageType = "pypi"
|
||||
|
||||
func init() { registerPackageType(PackagePyPI) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageRPM PackageType = "rpm"
|
||||
|
||||
func init() { registerPackageType(PackageRPM) }
|
||||
@@ -0,0 +1,5 @@
|
||||
package models
|
||||
|
||||
const PackageTerraform PackageType = "terraform"
|
||||
|
||||
func init() { registerPackageType(PackageTerraform) }
|
||||
@@ -1,33 +1,54 @@
|
||||
package models_test
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.unkin.net/unkin/artifactapi/pkg/models"
|
||||
)
|
||||
|
||||
// TestPackageTypeValid parses every package_type_<type>.go and asserts the
|
||||
// const it declares is registered, so a type file missing its init() fails.
|
||||
func TestPackageTypeValid(t *testing.T) {
|
||||
valid := []models.PackageType{
|
||||
models.PackageGeneric,
|
||||
models.PackageDocker,
|
||||
models.PackageHelm,
|
||||
models.PackagePyPI,
|
||||
models.PackageNPM,
|
||||
models.PackageRPM,
|
||||
models.PackageAlpine,
|
||||
models.PackagePuppet,
|
||||
models.PackageTerraform,
|
||||
models.PackageGoProxy,
|
||||
models.PackageCargo,
|
||||
models.PackageNuGet,
|
||||
models.PackageGitHubRPM,
|
||||
models.PackageGitHubDeb,
|
||||
models.PackageGitHubAlpine,
|
||||
files, err := filepath.Glob("package_type_*.go")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, pt := range valid {
|
||||
if !pt.Valid() {
|
||||
t.Errorf("expected %q to be valid", pt)
|
||||
seen := 0
|
||||
for _, f := range files {
|
||||
if strings.HasSuffix(f, "_test.go") {
|
||||
continue
|
||||
}
|
||||
file, err := parser.ParseFile(token.NewFileSet(), f, nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, decl := range file.Decls {
|
||||
gd, ok := decl.(*ast.GenDecl)
|
||||
if !ok || gd.Tok != token.CONST {
|
||||
continue
|
||||
}
|
||||
for _, spec := range gd.Specs {
|
||||
for _, v := range spec.(*ast.ValueSpec).Values {
|
||||
lit := v.(*ast.BasicLit)
|
||||
pt := models.PackageType(strings.Trim(lit.Value, `"`))
|
||||
if !pt.Valid() {
|
||||
t.Errorf("%s: %q is not registered", f, pt)
|
||||
}
|
||||
if want := "package_type_" + pt.String() + ".go"; f != want {
|
||||
t.Errorf("%s declares %q, want it in %s", f, pt, want)
|
||||
}
|
||||
seen++
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if seen == 0 {
|
||||
t.Fatal("found no package types")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,17 +1,7 @@
|
||||
import { useState } from 'react';
|
||||
import './UsageInstructions.css';
|
||||
|
||||
// repoClass distinguishes the three ways a repository is consumed. remotes are
|
||||
// caching proxies, locals are real registries you also publish to, virtuals are
|
||||
// merged read-only indexes.
|
||||
type RepoClass = 'remote' | 'local' | 'virtual';
|
||||
|
||||
interface Snippet {
|
||||
title: string;
|
||||
language: string;
|
||||
code: string;
|
||||
note?: string;
|
||||
}
|
||||
import { packageTypes } from '../packagetypes';
|
||||
import type { RepoClass, Snippet } from '../packagetypes/types';
|
||||
|
||||
// baseURL resolves the externally reachable origin of this artifactapi instance.
|
||||
// The UI is served on the same origin as the API (client BASE is ''), so
|
||||
@@ -46,315 +36,8 @@ export function buildSnippets(packageType: string, repoClass: RepoClass, name: s
|
||||
const proxy = remoteProxyBase(repoClass, name);
|
||||
const isLocal = repoClass === 'local';
|
||||
|
||||
switch (packageType) {
|
||||
case 'rpm':
|
||||
return [
|
||||
{
|
||||
title: isLocal
|
||||
? 'Add the yum repo (real yum repo, repodata auto-regenerated)'
|
||||
: 'Add the yum repo (caching proxy)',
|
||||
language: 'bash',
|
||||
code: `sudo tee /etc/yum.repos.d/${name}.repo >/dev/null <<'EOF'
|
||||
[${name}]
|
||||
name=${name} (artifactapi)
|
||||
baseurl=${isLocal ? `${url}/api/v2/remotes/${name}/files/` : `${proxy}/`}
|
||||
enabled=1
|
||||
gpgcheck=0
|
||||
repo_gpgcheck=0
|
||||
EOF
|
||||
|
||||
sudo dnf install <package>`,
|
||||
note: isLocal
|
||||
? 'gpgcheck=0: artifactapi serves the repo unsigned. If you sign your RPMs, import your key and set gpgcheck=1.'
|
||||
: 'gpgcheck=0 trusts upstream over the proxy. To verify package signatures, import the upstream GPG key and set gpgcheck=1.',
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish an RPM (repodata regenerates automatically)',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./my-package-1.0-1.el9.x86_64.rpm \\
|
||||
${url}/api/v2/remotes/${name}/files/my-package-1.0-1.el9.x86_64.rpm`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
|
||||
case 'pypi':
|
||||
return [
|
||||
{
|
||||
title: 'Install a package (one-off)',
|
||||
language: 'bash',
|
||||
code: `pip install --index-url ${proxy}/simple/ <package>`,
|
||||
},
|
||||
{
|
||||
title: 'Configure pip persistently',
|
||||
language: 'bash',
|
||||
code: `mkdir -p ~/.config/pip
|
||||
cat > ~/.config/pip/pip.conf <<'EOF'
|
||||
[global]
|
||||
index-url = ${proxy}/simple/
|
||||
EOF
|
||||
|
||||
pip install <package>`,
|
||||
},
|
||||
];
|
||||
|
||||
case 'npm':
|
||||
return [
|
||||
{
|
||||
title: 'Point npm at this registry',
|
||||
language: 'bash',
|
||||
code: `npm config set registry ${proxy}/
|
||||
npm install <package>`,
|
||||
},
|
||||
{
|
||||
title: 'Per-project (.npmrc)',
|
||||
language: 'bash',
|
||||
code: `echo 'registry=${proxy}/' >> .npmrc
|
||||
npm install`,
|
||||
},
|
||||
];
|
||||
|
||||
case 'docker':
|
||||
return [
|
||||
{
|
||||
title: 'Pull an image',
|
||||
language: 'bash',
|
||||
code: `docker pull ${host}/${name}/<image>:<tag>`,
|
||||
note: 'The first path segment after the host is the artifactapi repo name; the rest is the image name.',
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Push an image (this is a real Registry V2)',
|
||||
language: 'bash',
|
||||
code: `docker tag myapp:latest ${host}/${name}/myapp:latest
|
||||
docker push ${host}/${name}/myapp:latest`,
|
||||
note: 'Works with docker, podman, skopeo and buildah. If the registry requires auth, run `docker login ' + host + '` first.',
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
|
||||
case 'terraform':
|
||||
return [
|
||||
{
|
||||
title: 'Use as a provider source (bare address, no mirror config)',
|
||||
language: 'hcl',
|
||||
code: `terraform {
|
||||
required_providers {
|
||||
${name} = {
|
||||
source = "${host}/${name}/<type>"
|
||||
version = ">= 0.1.0"
|
||||
}
|
||||
}
|
||||
}`,
|
||||
note: 'The namespace segment is this repo name; <type> is the provider type. artifactapi signs SHA256SUMS server-side with its GPG key, so `terraform init` installs with no .terraformrc.',
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a provider build',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file terraform-provider-<type>_0.1.0_linux_amd64.zip \\
|
||||
${url}/api/v2/remotes/${name}/files/${name}/<type>/terraform-provider-<type>_0.1.0_linux_amd64.zip`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
|
||||
case 'helm':
|
||||
return [
|
||||
{
|
||||
title: isLocal ? 'Add the Helm repo (index.yaml auto-generated)' : 'Add the Helm repo',
|
||||
language: 'bash',
|
||||
code: `helm repo add ${name} ${isLocal ? `${url}/api/v1/local/${name}` : proxy}/
|
||||
helm repo update
|
||||
helm install <release> ${name}/<chart>`,
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a chart',
|
||||
language: 'bash',
|
||||
code: `helm package ./mychart
|
||||
curl -fsSL --upload-file mychart-0.1.0.tgz \\
|
||||
${url}/api/v2/remotes/${name}/files/mychart-0.1.0.tgz`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
|
||||
case 'alpine':
|
||||
return isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Add the apk repo (real apk repo, APKINDEX auto-generated)',
|
||||
language: 'bash',
|
||||
code: `echo '${url}/api/v1/local/${name}' | sudo tee -a /etc/apk/repositories
|
||||
sudo apk update --allow-untrusted
|
||||
sudo apk add --allow-untrusted <package>`,
|
||||
note: `Served unsigned (parity with the rpm repo's gpgcheck=0) — use --allow-untrusted, or install a signing key. apk fetches <arch>/APKINDEX.tar.gz under this base.`,
|
||||
},
|
||||
{
|
||||
title: 'Publish a .apk (index regenerates automatically)',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./mypkg-1.0-r0.apk \\
|
||||
${url}/api/v2/remotes/${name}/files/x86_64/mypkg-1.0-r0.apk`,
|
||||
note: 'Upload each package at <arch>/<name>-<version>.apk — apk reconstructs that exact path from the index (APKINDEX carries no filename), so a mismatched path will 404 on install.',
|
||||
},
|
||||
]
|
||||
: [
|
||||
{
|
||||
title: 'Add the APK repository',
|
||||
language: 'bash',
|
||||
code: `echo '${proxy}/' | sudo tee -a /etc/apk/repositories
|
||||
sudo apk update
|
||||
sudo apk add <package>`,
|
||||
note: 'If the index is unsigned over the proxy, add --allow-untrusted or install the signing key into /etc/apk/keys.',
|
||||
},
|
||||
];
|
||||
|
||||
case 'github_alpine':
|
||||
return [
|
||||
{
|
||||
title: 'Add the apk repo (metadata-only, from GitHub releases)',
|
||||
language: 'bash',
|
||||
code: `echo '${proxy}' | sudo tee -a /etc/apk/repositories
|
||||
sudo apk update --allow-untrusted
|
||||
sudo apk add --allow-untrusted <package>`,
|
||||
note: "The per-arch APKINDEX is synthesized from the configured GitHub repo's release .apk assets; package downloads are redirected to the backing releases remote. Served unsigned, so --allow-untrusted.",
|
||||
},
|
||||
];
|
||||
|
||||
case 'goproxy':
|
||||
return [
|
||||
{
|
||||
title: 'Point the Go module proxy here',
|
||||
language: 'bash',
|
||||
code: `export GOPROXY=${proxy}
|
||||
go mod download`,
|
||||
note: 'Append ,direct to fall back to VCS for modules this proxy does not cover.',
|
||||
},
|
||||
];
|
||||
|
||||
case 'cargo':
|
||||
return [
|
||||
{
|
||||
title: 'Point crates.io at this sparse registry',
|
||||
language: 'toml',
|
||||
code: `# ~/.cargo/config.toml
|
||||
[source.crates-io]
|
||||
replace-with = "${name}"
|
||||
|
||||
[registries.${name}]
|
||||
index = "sparse+${proxy}/"`,
|
||||
note: 'config.json is served by artifactapi so crate downloads also go through this remote.',
|
||||
},
|
||||
];
|
||||
|
||||
case 'nuget':
|
||||
return [
|
||||
{
|
||||
title: 'Use this feed instead of nuget.org',
|
||||
language: 'xml',
|
||||
code: `<!-- nuget.config -->
|
||||
<configuration>
|
||||
<packageSources>
|
||||
<clear />
|
||||
<add key="${name}" value="${proxy}/index.json" protocolVersion="3" />
|
||||
</packageSources>
|
||||
</configuration>`,
|
||||
note: 'Serve artifactapi over HTTPS: NuGet rejects the repository signatures resource over plain HTTP.',
|
||||
},
|
||||
];
|
||||
|
||||
case 'puppet':
|
||||
return [
|
||||
{
|
||||
title: isLocal ? 'Install a module from this forge' : 'Install a module from the Forge proxy',
|
||||
language: 'bash',
|
||||
code: `puppet module install <author>-<module> \\
|
||||
--module_repository ${isLocal ? `${url}/api/v1/local/${name}` : proxy}`,
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a module (versions are immutable)',
|
||||
language: 'bash',
|
||||
code: `puppet module build && curl -fsSL --upload-file pkg/<author>-<module>-<version>.tar.gz \\
|
||||
${url}/api/v2/remotes/${name}/files/<author>-<module>-<version>.tar.gz`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
|
||||
case 'deb':
|
||||
return isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Add the apt repo (real apt repo, flat — Packages/Release auto-generated)',
|
||||
language: 'bash',
|
||||
code: `echo 'deb [trusted=yes] ${url}/api/v1/local/${name}/ ./' | sudo tee /etc/apt/sources.list.d/${name}.list
|
||||
sudo apt-get update
|
||||
sudo apt-get install <package>`,
|
||||
note: '[trusted=yes]: artifactapi serves the flat repo unsigned (matches the rpm repo\'s gpgcheck=0). The `./` is the flat-repo suite — apt fetches Packages/Release from the repo root.',
|
||||
},
|
||||
{
|
||||
title: 'Publish a .deb (index regenerates automatically)',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./my-package_1.0_amd64.deb \\
|
||||
${url}/api/v2/remotes/${name}/files/my-package_1.0_amd64.deb`,
|
||||
},
|
||||
]
|
||||
: [
|
||||
{
|
||||
title: 'Add the apt repo (caching proxy)',
|
||||
language: 'bash',
|
||||
code: `echo 'deb ${proxy} <suite> <component>' | sudo tee /etc/apt/sources.list.d/${name}.list
|
||||
sudo apt-get update
|
||||
sudo apt-get install <package>`,
|
||||
note: "Signatures are verified against the upstream mirror's real signed Release through the proxy (no [trusted=yes] needed). Example suite/component: bookworm main.",
|
||||
},
|
||||
];
|
||||
|
||||
case 'github_deb':
|
||||
return [
|
||||
{
|
||||
title: 'Add the apt repo (metadata-only, from GitHub releases)',
|
||||
language: 'bash',
|
||||
code: `echo 'deb [trusted=yes] ${proxy}/ ./' | sudo tee /etc/apt/sources.list.d/${name}.list
|
||||
sudo apt-get update
|
||||
sudo apt-get install <package>`,
|
||||
note: "The apt index is synthesized from the configured GitHub repo's release .deb assets; package downloads are redirected to the backing releases remote. Served unsigned, so [trusted=yes].",
|
||||
},
|
||||
];
|
||||
|
||||
case 'generic':
|
||||
default:
|
||||
return [
|
||||
{
|
||||
title: 'Download a file',
|
||||
language: 'bash',
|
||||
code: `curl -fsSLO ${proxy}/<path>`,
|
||||
note:
|
||||
packageType === 'generic'
|
||||
? 'Generic repos are fetched as plain files at their upstream path.'
|
||||
: `No tailored client instructions for "${packageType}" yet — fetch artifacts directly by path.`,
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a file',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./myfile \\
|
||||
${url}/api/v2/remotes/${name}/files/<path>/myfile`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
}
|
||||
const def = packageTypes[packageType] ?? packageTypes.generic;
|
||||
return def.snippets({ packageType, name, url, host, proxy, isLocal });
|
||||
}
|
||||
|
||||
function CodeBox({ snippet }: { snippet: Snippet }) {
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'green',
|
||||
snippets: ({ name, url, proxy, isLocal }) => {
|
||||
return isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Add the apk repo (real apk repo, APKINDEX auto-generated)',
|
||||
language: 'bash',
|
||||
code: `echo '${url}/api/v1/local/${name}' | sudo tee -a /etc/apk/repositories
|
||||
sudo apk update --allow-untrusted
|
||||
sudo apk add --allow-untrusted <package>`,
|
||||
note: `Served unsigned (parity with the rpm repo's gpgcheck=0) — use --allow-untrusted, or install a signing key. apk fetches <arch>/APKINDEX.tar.gz under this base.`,
|
||||
},
|
||||
{
|
||||
title: 'Publish a .apk (index regenerates automatically)',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./mypkg-1.0-r0.apk \\
|
||||
${url}/api/v2/remotes/${name}/files/x86_64/mypkg-1.0-r0.apk`,
|
||||
note: 'Upload each package at <arch>/<name>-<version>.apk — apk reconstructs that exact path from the index (APKINDEX carries no filename), so a mismatched path will 404 on install.',
|
||||
},
|
||||
]
|
||||
: [
|
||||
{
|
||||
title: 'Add the APK repository',
|
||||
language: 'bash',
|
||||
code: `echo '${proxy}/' | sudo tee -a /etc/apk/repositories
|
||||
sudo apk update
|
||||
sudo apk add <package>`,
|
||||
note: 'If the index is unsigned over the proxy, add --allow-untrusted or install the signing key into /etc/apk/keys.',
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,22 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'yellow',
|
||||
snippets: ({ name, proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Point crates.io at this sparse registry',
|
||||
language: 'toml',
|
||||
code: `# ~/.cargo/config.toml
|
||||
[source.crates-io]
|
||||
replace-with = "${name}"
|
||||
|
||||
[registries.${name}]
|
||||
index = "sparse+${proxy}/"`,
|
||||
note: 'config.json is served by artifactapi so crate downloads also go through this remote.',
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,35 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
snippets: ({ name, url, proxy, isLocal }) => {
|
||||
return isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Add the apt repo (real apt repo, flat — Packages/Release auto-generated)',
|
||||
language: 'bash',
|
||||
code: `echo 'deb [trusted=yes] ${url}/api/v1/local/${name}/ ./' | sudo tee /etc/apt/sources.list.d/${name}.list
|
||||
sudo apt-get update
|
||||
sudo apt-get install <package>`,
|
||||
note: '[trusted=yes]: artifactapi serves the flat repo unsigned (matches the rpm repo\'s gpgcheck=0). The `./` is the flat-repo suite — apt fetches Packages/Release from the repo root.',
|
||||
},
|
||||
{
|
||||
title: 'Publish a .deb (index regenerates automatically)',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./my-package_1.0_amd64.deb \\
|
||||
${url}/api/v2/remotes/${name}/files/my-package_1.0_amd64.deb`,
|
||||
},
|
||||
]
|
||||
: [
|
||||
{
|
||||
title: 'Add the apt repo (caching proxy)',
|
||||
language: 'bash',
|
||||
code: `echo 'deb ${proxy} <suite> <component>' | sudo tee /etc/apt/sources.list.d/${name}.list
|
||||
sudo apt-get update
|
||||
sudo apt-get install <package>`,
|
||||
note: "Signatures are verified against the upstream mirror's real signed Release through the proxy (no [trusted=yes] needed). Example suite/component: bookworm main.",
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,28 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'blue',
|
||||
snippets: ({ name, host, isLocal }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Pull an image',
|
||||
language: 'bash',
|
||||
code: `docker pull ${host}/${name}/<image>:<tag>`,
|
||||
note: 'The first path segment after the host is the artifactapi repo name; the rest is the image name.',
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Push an image (this is a real Registry V2)',
|
||||
language: 'bash',
|
||||
code: `docker tag myapp:latest ${host}/${name}/myapp:latest
|
||||
docker push ${host}/${name}/myapp:latest`,
|
||||
note: 'Works with docker, podman, skopeo and buildah. If the registry requires auth, run `docker login ' + host + '` first.',
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,30 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'default',
|
||||
snippets: ({ packageType, name, url, proxy, isLocal }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Download a file',
|
||||
language: 'bash',
|
||||
code: `curl -fsSLO ${proxy}/<path>`,
|
||||
note:
|
||||
packageType === 'generic'
|
||||
? 'Generic repos are fetched as plain files at their upstream path.'
|
||||
: `No tailored client instructions for "${packageType}" yet — fetch artifacts directly by path.`,
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a file',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./myfile \\
|
||||
${url}/api/v2/remotes/${name}/files/<path>/myfile`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
snippets: ({ proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Add the apk repo (metadata-only, from GitHub releases)',
|
||||
language: 'bash',
|
||||
code: `echo '${proxy}' | sudo tee -a /etc/apk/repositories
|
||||
sudo apk update --allow-untrusted
|
||||
sudo apk add --allow-untrusted <package>`,
|
||||
note: "The per-arch APKINDEX is synthesized from the configured GitHub repo's release .apk assets; package downloads are redirected to the backing releases remote. Served unsigned, so --allow-untrusted.",
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
snippets: ({ name, proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Add the apt repo (metadata-only, from GitHub releases)',
|
||||
language: 'bash',
|
||||
code: `echo 'deb [trusted=yes] ${proxy}/ ./' | sudo tee /etc/apt/sources.list.d/${name}.list
|
||||
sudo apt-get update
|
||||
sudo apt-get install <package>`,
|
||||
note: "The apt index is synthesized from the configured GitHub repo's release .deb assets; package downloads are redirected to the backing releases remote. Served unsigned, so [trusted=yes].",
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'green',
|
||||
snippets: ({ proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Point the Go module proxy here',
|
||||
language: 'bash',
|
||||
code: `export GOPROXY=${proxy}
|
||||
go mod download`,
|
||||
note: 'Append ,direct to fall back to VCS for modules this proxy does not cover.',
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,29 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'green',
|
||||
snippets: ({ name, url, proxy, isLocal }) => {
|
||||
return [
|
||||
{
|
||||
title: isLocal ? 'Add the Helm repo (index.yaml auto-generated)' : 'Add the Helm repo',
|
||||
language: 'bash',
|
||||
code: `helm repo add ${name} ${isLocal ? `${url}/api/v1/local/${name}` : proxy}/
|
||||
helm repo update
|
||||
helm install <release> ${name}/<chart>`,
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a chart',
|
||||
language: 'bash',
|
||||
code: `helm package ./mychart
|
||||
curl -fsSL --upload-file mychart-0.1.0.tgz \\
|
||||
${url}/api/v2/remotes/${name}/files/mychart-0.1.0.tgz`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'red',
|
||||
snippets: ({ proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Point npm at this registry',
|
||||
language: 'bash',
|
||||
code: `npm config set registry ${proxy}/
|
||||
npm install <package>`,
|
||||
},
|
||||
{
|
||||
title: 'Per-project (.npmrc)',
|
||||
language: 'bash',
|
||||
code: `echo 'registry=${proxy}/' >> .npmrc
|
||||
npm install`,
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'blue',
|
||||
snippets: ({ name, proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Use this feed instead of nuget.org',
|
||||
language: 'xml',
|
||||
code: `<!-- nuget.config -->
|
||||
<configuration>
|
||||
<packageSources>
|
||||
<clear />
|
||||
<add key="${name}" value="${proxy}/index.json" protocolVersion="3" />
|
||||
</packageSources>
|
||||
</configuration>`,
|
||||
note: 'Serve artifactapi over HTTPS: NuGet rejects the repository signatures resource over plain HTTP.',
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'yellow',
|
||||
snippets: ({ name, url, proxy, isLocal }) => {
|
||||
return [
|
||||
{
|
||||
title: isLocal ? 'Install a module from this forge' : 'Install a module from the Forge proxy',
|
||||
language: 'bash',
|
||||
code: `puppet module install <author>-<module> \\
|
||||
--module_repository ${isLocal ? `${url}/api/v1/local/${name}` : proxy}`,
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a module (versions are immutable)',
|
||||
language: 'bash',
|
||||
code: `puppet module build && curl -fsSL --upload-file pkg/<author>-<module>-<version>.tar.gz \\
|
||||
${url}/api/v2/remotes/${name}/files/<author>-<module>-<version>.tar.gz`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'blue',
|
||||
snippets: ({ proxy }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Install a package (one-off)',
|
||||
language: 'bash',
|
||||
code: `pip install --index-url ${proxy}/simple/ <package>`,
|
||||
},
|
||||
{
|
||||
title: 'Configure pip persistently',
|
||||
language: 'bash',
|
||||
code: `mkdir -p ~/.config/pip
|
||||
cat > ~/.config/pip/pip.conf <<'EOF'
|
||||
[global]
|
||||
index-url = ${proxy}/simple/
|
||||
EOF
|
||||
|
||||
pip install <package>`,
|
||||
},
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,40 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'yellow',
|
||||
snippets: ({ name, url, proxy, isLocal }) => {
|
||||
return [
|
||||
{
|
||||
title: isLocal
|
||||
? 'Add the yum repo (real yum repo, repodata auto-regenerated)'
|
||||
: 'Add the yum repo (caching proxy)',
|
||||
language: 'bash',
|
||||
code: `sudo tee /etc/yum.repos.d/${name}.repo >/dev/null <<'EOF'
|
||||
[${name}]
|
||||
name=${name} (artifactapi)
|
||||
baseurl=${isLocal ? `${url}/api/v2/remotes/${name}/files/` : `${proxy}/`}
|
||||
enabled=1
|
||||
gpgcheck=0
|
||||
repo_gpgcheck=0
|
||||
EOF
|
||||
|
||||
sudo dnf install <package>`,
|
||||
note: isLocal
|
||||
? 'gpgcheck=0: artifactapi serves the repo unsigned. If you sign your RPMs, import your key and set gpgcheck=1.'
|
||||
: 'gpgcheck=0 trusts upstream over the proxy. To verify package signatures, import the upstream GPG key and set gpgcheck=1.',
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish an RPM (repodata regenerates automatically)',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file ./my-package-1.0-1.el9.x86_64.rpm \\
|
||||
${url}/api/v2/remotes/${name}/files/my-package-1.0-1.el9.x86_64.rpm`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,34 @@
|
||||
import type { PackageTypeUI } from '../types';
|
||||
|
||||
const def: PackageTypeUI = {
|
||||
color: 'blue',
|
||||
snippets: ({ name, url, host, isLocal }) => {
|
||||
return [
|
||||
{
|
||||
title: 'Use as a provider source (bare address, no mirror config)',
|
||||
language: 'hcl',
|
||||
code: `terraform {
|
||||
required_providers {
|
||||
${name} = {
|
||||
source = "${host}/${name}/<type>"
|
||||
version = ">= 0.1.0"
|
||||
}
|
||||
}
|
||||
}`,
|
||||
note: 'The namespace segment is this repo name; <type> is the provider type. artifactapi signs SHA256SUMS server-side with its GPG key, so `terraform init` installs with no .terraformrc.',
|
||||
},
|
||||
...(isLocal
|
||||
? [
|
||||
{
|
||||
title: 'Publish a provider build',
|
||||
language: 'bash',
|
||||
code: `curl -fsSL --upload-file terraform-provider-<type>_0.1.0_linux_amd64.zip \\
|
||||
${url}/api/v2/remotes/${name}/files/${name}/<type>/terraform-provider-<type>_0.1.0_linux_amd64.zip`,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default def;
|
||||
@@ -0,0 +1,11 @@
|
||||
import type { BadgeColor, PackageTypeUI } from './types';
|
||||
|
||||
const defs = import.meta.glob<PackageTypeUI>('./defs/*.ts', { eager: true, import: 'default' });
|
||||
|
||||
export const packageTypes: Record<string, PackageTypeUI> = Object.fromEntries(
|
||||
Object.entries(defs).map(([path, def]) => [path.slice('./defs/'.length, -'.ts'.length), def]),
|
||||
);
|
||||
|
||||
export function typeColor(packageType: string): BadgeColor {
|
||||
return packageTypes[packageType]?.color || 'default';
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
// repoClass distinguishes the three ways a repository is consumed. remotes are
|
||||
// caching proxies, locals are real registries you also publish to, virtuals are
|
||||
// merged read-only indexes.
|
||||
export type RepoClass = 'remote' | 'local' | 'virtual';
|
||||
|
||||
export interface Snippet {
|
||||
title: string;
|
||||
language: string;
|
||||
code: string;
|
||||
note?: string;
|
||||
}
|
||||
|
||||
export interface SnippetContext {
|
||||
packageType: string;
|
||||
name: string;
|
||||
url: string;
|
||||
host: string;
|
||||
proxy: string;
|
||||
isLocal: boolean;
|
||||
}
|
||||
|
||||
export type BadgeColor = 'blue' | 'green' | 'yellow' | 'red' | 'default';
|
||||
|
||||
export interface PackageTypeUI {
|
||||
color?: BadgeColor;
|
||||
snippets: (ctx: SnippetContext) => Snippet[];
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user