feat: use top-level key for repo type instead of type field
Replace the flat `remotes:` map (with `type: "remote"/"virtual"/"local"`) with separate top-level sections — `remote:`, `virtual:`, `local:` — so the repo type is declared structurally and the `type:` field is no longer needed. Config loader normalises the new format to the existing internal representation (injecting `type` into each remote dict), so all handler code is unchanged. Adds a TestYamlTypeKeys suite covering all three type keys, mixed files, and field preservation. Includes README migration guide for splitting a single remotes file into per-type-and-package conf.d files.
This commit is contained in:
@@ -122,11 +122,12 @@ remotes: {} # optional base remotes
|
||||
|
||||
### remotes.yaml Structure
|
||||
|
||||
The top-level key declares the repository type — no `type:` field needed:
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
remote-name:
|
||||
base_url: "https://example.com"
|
||||
type: "remote" # "remote", "local", or "virtual"
|
||||
package: "generic" # generic, alpine, rpm, docker, pypi, npm, helm
|
||||
description: "..."
|
||||
immutable_patterns: # regex — cached forever
|
||||
@@ -137,6 +138,17 @@ remotes:
|
||||
cache:
|
||||
immutable_ttl: 0 # 0 = indefinitely
|
||||
mutable_ttl: 3600
|
||||
|
||||
virtual:
|
||||
virtual-name:
|
||||
package: "helm"
|
||||
members:
|
||||
- remote-name-1
|
||||
- remote-name-2
|
||||
|
||||
local:
|
||||
local-name:
|
||||
package: "generic"
|
||||
```
|
||||
|
||||
## Remote Types
|
||||
@@ -146,10 +158,9 @@ remotes:
|
||||
Arbitrary HTTP file servers — GitHub releases, HashiCorp, custom servers.
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
github:
|
||||
base_url: "https://github.com"
|
||||
type: "remote"
|
||||
package: "generic"
|
||||
immutable_patterns:
|
||||
- "gruntwork-io/terragrunt/.*terragrunt_linux_amd64.*"
|
||||
@@ -158,7 +169,6 @@ remotes:
|
||||
|
||||
github-archive:
|
||||
base_url: "https://github.com"
|
||||
type: "remote"
|
||||
package: "generic"
|
||||
immutable_patterns:
|
||||
- ".*/archive/refs/tags/.*\\.tar\\.gz$" # tag archives never change
|
||||
@@ -175,10 +185,9 @@ Access: `GET /api/v1/remote/github/owner/repo/releases/download/v1.0/binary.tar.
|
||||
### alpine
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
alpine:
|
||||
base_url: "https://dl-cdn.alpinelinux.org"
|
||||
type: "remote"
|
||||
package: "alpine"
|
||||
immutable_patterns:
|
||||
- ".*/x86_64/.*\\.apk$"
|
||||
@@ -192,10 +201,9 @@ remotes:
|
||||
### rpm
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
almalinux:
|
||||
base_url: "https://mirror.example.com/almalinux"
|
||||
type: "remote"
|
||||
package: "rpm"
|
||||
immutable_patterns:
|
||||
- ".*/x86_64/.*\\.rpm$"
|
||||
@@ -210,10 +218,9 @@ remotes:
|
||||
### docker
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
dockerhub:
|
||||
base_url: "https://registry-1.docker.io"
|
||||
type: "remote"
|
||||
package: "docker"
|
||||
# username / password optional for public images
|
||||
cache:
|
||||
@@ -222,7 +229,6 @@ remotes:
|
||||
|
||||
ghcr:
|
||||
base_url: "https://ghcr.io"
|
||||
type: "remote"
|
||||
package: "docker"
|
||||
username: "your-github-username"
|
||||
password: "ghp_your_pat" # read:packages scope
|
||||
@@ -252,10 +258,9 @@ mirrors:
|
||||
### pypi
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
pypi:
|
||||
base_url: "https://files.pythonhosted.org"
|
||||
type: "remote"
|
||||
package: "pypi"
|
||||
check_mutable_updates: true
|
||||
immutable_patterns:
|
||||
@@ -284,10 +289,9 @@ default = true
|
||||
### npm
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
npm:
|
||||
base_url: "https://registry.npmjs.org"
|
||||
type: "remote"
|
||||
package: "npm"
|
||||
check_mutable_updates: true
|
||||
immutable_patterns:
|
||||
@@ -311,10 +315,9 @@ registry=https://artifacts.example.com/api/v1/remote/npm/
|
||||
### helm
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
hashicorp-helm:
|
||||
base_url: "https://helm.releases.hashicorp.com"
|
||||
type: "remote"
|
||||
package: "helm"
|
||||
check_mutable_updates: true
|
||||
immutable_patterns:
|
||||
@@ -340,10 +343,9 @@ A virtual repository presents a single unified index built from multiple member
|
||||
All members must share the same `package` type as the virtual repo. Currently supported package types: `helm`.
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
helm-hashicorp:
|
||||
base_url: "https://helm.releases.hashicorp.com"
|
||||
type: "remote"
|
||||
package: "helm"
|
||||
immutable_patterns:
|
||||
- "\\.tgz$"
|
||||
@@ -353,7 +355,6 @@ remotes:
|
||||
|
||||
helm-bitnami:
|
||||
base_url: "https://charts.bitnami.com/bitnami"
|
||||
type: "remote"
|
||||
package: "helm"
|
||||
immutable_patterns:
|
||||
- "\\.tgz$"
|
||||
@@ -361,8 +362,8 @@ remotes:
|
||||
immutable_ttl: 0
|
||||
mutable_ttl: 3600
|
||||
|
||||
virtual:
|
||||
helm-all:
|
||||
type: "virtual"
|
||||
package: "helm"
|
||||
members:
|
||||
- helm-hashicorp # listed first = highest priority
|
||||
@@ -399,9 +400,8 @@ Chart tarball URLs in the merged `index.yaml` are rewritten to point at the indi
|
||||
### local
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
local:
|
||||
local-generic:
|
||||
type: "local"
|
||||
package: "generic"
|
||||
description: "Local file repository"
|
||||
cache:
|
||||
@@ -411,6 +411,98 @@ remotes:
|
||||
|
||||
No `base_url`. Files are uploaded via `PUT` and served via `GET`.
|
||||
|
||||
## Migration
|
||||
|
||||
### Splitting a single remotes file into per-type files
|
||||
|
||||
The old format used a single `remotes:` map with an explicit `type:` field on each entry. The new format uses top-level type keys (`remote:`, `virtual:`, `local:`) and supports splitting across multiple files via `config_dir`.
|
||||
|
||||
**Before** (`remotes.yaml`):
|
||||
```yaml
|
||||
remotes:
|
||||
dockerhub:
|
||||
base_url: "https://registry-1.docker.io"
|
||||
type: "remote"
|
||||
package: "docker"
|
||||
cache:
|
||||
immutable_ttl: 0
|
||||
mutable_ttl: 300
|
||||
|
||||
hashicorp-helm:
|
||||
base_url: "https://helm.releases.hashicorp.com"
|
||||
type: "remote"
|
||||
package: "helm"
|
||||
immutable_patterns:
|
||||
- "\\.tgz$"
|
||||
cache:
|
||||
immutable_ttl: 0
|
||||
mutable_ttl: 3600
|
||||
|
||||
helm-all:
|
||||
type: "virtual"
|
||||
package: "helm"
|
||||
members:
|
||||
- hashicorp-helm
|
||||
|
||||
local-files:
|
||||
type: "local"
|
||||
package: "generic"
|
||||
```
|
||||
|
||||
**After** — one file per type + package type, with a main config pointing at the directory:
|
||||
|
||||
`config.yaml`:
|
||||
```yaml
|
||||
config_dir: conf.d
|
||||
```
|
||||
|
||||
`conf.d/remote-docker.yaml`:
|
||||
```yaml
|
||||
remote:
|
||||
dockerhub:
|
||||
base_url: "https://registry-1.docker.io"
|
||||
package: "docker"
|
||||
cache:
|
||||
immutable_ttl: 0
|
||||
mutable_ttl: 300
|
||||
```
|
||||
|
||||
`conf.d/remote-helm.yaml`:
|
||||
```yaml
|
||||
remote:
|
||||
hashicorp-helm:
|
||||
base_url: "https://helm.releases.hashicorp.com"
|
||||
package: "helm"
|
||||
immutable_patterns:
|
||||
- "\\.tgz$"
|
||||
cache:
|
||||
immutable_ttl: 0
|
||||
mutable_ttl: 3600
|
||||
```
|
||||
|
||||
`conf.d/virtual-helm.yaml`:
|
||||
```yaml
|
||||
virtual:
|
||||
helm-all:
|
||||
package: "helm"
|
||||
members:
|
||||
- hashicorp-helm
|
||||
```
|
||||
|
||||
`conf.d/local-generic.yaml`:
|
||||
```yaml
|
||||
local:
|
||||
local-files:
|
||||
package: "generic"
|
||||
```
|
||||
|
||||
Set `CONFIG_PATH` to the main file:
|
||||
```
|
||||
CONFIG_PATH=/etc/artifactapi/config.yaml
|
||||
```
|
||||
|
||||
Files in `conf.d/` are merged alphabetically; later files win on conflicts within the same remote name.
|
||||
|
||||
## Caching Model
|
||||
|
||||
### Immutable patterns
|
||||
@@ -448,10 +540,9 @@ When a mutable file expires and the upstream is unreachable (connection refused,
|
||||
Set `quarantine_new: true` and `quarantine_days: N` on a remote to block immutable artifacts published within the last N days. Requests return `404` until the quarantine period expires, giving time to detect malicious packages before they are consumed.
|
||||
|
||||
```yaml
|
||||
remotes:
|
||||
remote:
|
||||
pypi:
|
||||
base_url: "https://files.pythonhosted.org"
|
||||
type: "remote"
|
||||
package: "pypi"
|
||||
quarantine_new: true
|
||||
quarantine_days: 3 # block packages published in the last 3 days
|
||||
|
||||
Reference in New Issue
Block a user