A remote's base_url may now be a single string OR a list of upstream
mirrors. When it is a list the shared proxy engine load-balances across
them round-robin and, on an upstream error/timeout/5xx, fails over to the
next mirror before returning an error. Because the selection happens in
the engine (not per provider), it applies to every remote package type.
Backward compatible: a bare-string base_url behaves exactly as before.
- add models.StringOrSlice (string-or-array JSON) and custom Remote
(Un)MarshalJSON: base_url populates BaseURLs (full list) + BaseURL
(active/first); marshals a single mirror back to a bare string
- add Remote.BaseURLList / ValidateBaseURLs; validate list is non-empty
and every entry is an http/https URL in the v2 create/update handlers
- persist the full list in a new base_urls TEXT[] column (additive
migration), keeping base_url in sync for old readers; only write
base_urls for genuinely multi-mirror remotes
- engine: per-remote round-robin cursor + attempt ordering; wrap the
fetch/head/revalidate upstream calls in a failover loop that narrows
the remote to one selected mirror per attempt; only network errors and
5xx fail over (404/403/... are returned as-is); circuit breaker stays
keyed per remote and trips only after all mirrors fail
- add unit tests (JSON round-trip, engine round-robin/failover/single-URL,
DB multi-URL round-trip) and a docker acceptance suite: round-robin
distribution across two mock upstreams, failover past a dead primary,
single-base_url regression, and a real dnf makecache+install through a
two-mirror rpm remote whose primary is dead
Least-connections and a per-remote strategy selector are a follow-up PR.
Fixes#67
## Why
The proxy used `http.DefaultClient` for all upstream GET/HEAD and bearer-token requests. It has no timeouts, so a slow or hung upstream holds a goroutine and connection indefinitely.
## Changes
- Add a shared `upstreamClient` (`internal/proxy/httpclient.go`) with dial, TLS-handshake, response-header and idle-connection timeouts, plus connection pooling.
- Deliberately no overall `Client.Timeout`, so large artifact bodies can still stream; total time is bounded by the request context.
- Route all four upstream calls in the engine through it.
## Validation
- `make e2e` passes.
Reviewed-on: #83
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#72
## Why
`compilePatterns` silently discards any pattern that fails to compile. A typo in a blocklist entry therefore turns a deny rule into a no-op — a fail-open with security impact.
## Changes
- Add `Remote.ValidatePatterns`, which compiles every pattern list (patterns, blocklist, mutable/immutable patterns, ban_tags) and returns an error on the first invalid regex.
- Reject invalid patterns with 400 at remote create and update time.
- Unit test for valid and invalid patterns.
## Validation
- `go test ./pkg/models/` and `make e2e` pass.
Reviewed-on: #87
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Introduces repo_type (remote/local) as a separate axis from package_type
so that any package type can be hosted locally. A terraform local repo
is package_type=terraform + repo_type=local.
- Remote model gains RepoType field (defaults to "remote")
- Database schema adds repo_type column with migration for existing DBs
- V1 proxy adds /api/v1/local/{name}/* route for serving local files
- V2 upload via PUT /api/v2/remotes/{name}/files/{ns}/{type}/{file}.zip
validates filename matches terraform-provider-{type}_{ver}_{os}_{arch}.zip
and returns 409 on duplicate (no overwrites)
- index.json and {version}.json are computed on-the-fly from uploaded zips
rather than stored as separate files
- V2 create validates repo_type and requires base_url only for remotes
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #49
unkin-agent
unkinben
benvin