## Why
`master` does not compile. Three PRs that each built individually combined badly:
- #92 changed `fetchBearerToken` to return `(string, time.Duration, error)` and added `cachedBearerToken` (which hashes the challenge via `sha256Hash`).
- #94 (streaming) removed the now-unused-in-that-PR `sha256Hash` helper and its `crypto/sha256` / `encoding/hex` imports.
- #89 (HEAD) added `headUpstream`, which calls `fetchBearerToken` expecting two return values.
Result on `master`: `internal/proxy/engine.go` fails to build (`assignment mismatch: 2 variables but fetchBearerToken returns 3 values`; `undefined: sha256Hash`).
## Changes
- Re-add the `sha256Hash` helper and its `crypto/sha256` + `encoding/hex` imports.
- Fix the `headUpstream` 401 path to handle `fetchBearerToken`s three return values.
## Validation
- `go build ./...`, `go vet`, and `make e2e` all pass.
Should merge before the other in-flight branches so they rebase onto a compiling `master`.
Reviewed-on: #96
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#67
## Why
The proxy used `http.DefaultClient` for all upstream GET/HEAD and bearer-token requests. It has no timeouts, so a slow or hung upstream holds a goroutine and connection indefinitely.
## Changes
- Add a shared `upstreamClient` (`internal/proxy/httpclient.go`) with dial, TLS-handshake, response-header and idle-connection timeouts, plus connection pooling.
- Deliberately no overall `Client.Timeout`, so large artifact bodies can still stream; total time is bounded by the request context.
- Route all four upstream calls in the engine through it.
## Validation
- `make e2e` passes.
Reviewed-on: #83
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#69
## Why
`getenv` returned the fallback whenever `os.Getenv` was empty, so an intentionally-empty env var could not override a non-empty default.
## Changes
- Use `os.LookupEnv` to distinguish unset from set-but-empty.
## Validation
- `make e2e` passes.
Reviewed-on: #85
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#75
## Why
On a fetch-lock miss, `Engine.Fetch` slept a flat 500ms once, tried the store, and otherwise fell through to fetch upstream unlocked. A cold-cache stampede therefore still hit upstream once per waiter.
## Changes
- Add `waitForStore`, which polls the store every 100ms for up to 5s (stopping on context cancellation) so waiters pick up the lock leaders populated result.
- Only fall through to an upstream fetch if the leader has not populated the store within the wait budget.
## Validation
- `make e2e` passes.
Reviewed-on: #93
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#76
## Why
Every proxied request spawned a goroutine running a 5s-timeout single-row INSERT. Under load this is unbounded goroutines and connection-pool pressure.
## Changes
- Add `database.AccessLogEntry` + `InsertAccessLogBatch` (bulk `COPY`).
- The engine starts one background writer that drains a buffered channel and flushes every 128 entries or 2s.
- `logAccess` is now a non-blocking channel send (drops on full buffer), so the request path never blocks on the DB. Best-effort telemetry: a small tail may be lost on abrupt shutdown.
## Validation
- `make e2e` passes.
Reviewed-on: #91
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#70
## Why
Docker `HEAD` routes mapped to `handleProxy`, which ran a full `Fetch` + `io.Copy` — downloading the entire blob (and fetching upstream on a miss) only for net/http to discard the body. HEAD existence checks (manifests, blobs) are common.
## Changes
- Add `Engine.Head`: answers cached artifacts/indexes from store metadata (no blob download); on a miss issues an upstream `HEAD` (with bearer-token handling) and never caches a body.
- Route `HEAD /v2/{remote}/*` to a dedicated `handleProxyHead` that writes headers only.
- Add e2e tests for HEAD on a blocklisted path (403) and an unknown remote (404).
## Note
`headUpstream` uses `http.DefaultClient` to build cleanly on master; it will pick up the shared timeout-configured client from #67 once that merges.
## Validation
- `make e2e` passes (includes new HEAD tests).
Reviewed-on: #89
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#77
## Why
Each upstream 401 re-ran the full token-endpoint request, even though a single Docker pull triggers many blob/manifest requests sharing one scope.
## Changes
- Add Redis `GetToken`/`SetToken`.
- `fetchBearerToken` now also parses `expires_in` and returns a TTL.
- New `Engine.cachedBearerToken` reuses a cached token keyed by remote + challenge (hashed), caching for `expires_in` minus a safety margin (default 60s when absent).
## Validation
- `make e2e` passes.
Reviewed-on: #92
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#66
## Why
`fetchFromUpstream` read every upstream response with `io.ReadAll`, hashed it in memory, uploaded from memory and served from memory. A single large immutable blob (Docker layer, RPM, tarball, Go module zip) — or several concurrent ones — could OOM the process. The streaming, tempfile-backed CAS already existed but the proxy path bypassed it (and `Engine.cas` was assigned but unused).
## Changes
- Immutable fetches now stream through `CAS.Store` (tempfile -> sha256 -> S3), so memory stays bounded regardless of artifact size, and are served back from the store.
- Mutable indexes stay on the in-memory path (small, and subject to `RewriteResponse`).
- Skipping `RewriteResponse` for immutable content is behaviour-preserving: the proxy path always passes an empty `proxyBaseURL`, under which every providers `RewriteResponse` is a no-op.
- Remove the now-unused in-memory `sha256Hash` helper.
## Validation
- `make e2e` passes.
- Live smoke test against Postgres/Redis/MinIO: proxied a 12 MB blob through a generic remote — fetch #1 `X-Artifact-Source: remote`, fetch #2 `X-Artifact-Source: cache`, both byte-identical (sha256) to the origin.
Reviewed-on: #94
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#73
## Why
`Classifier.Classify` runs on every proxied request and recompiled the Blocklist/Patterns/Immutable/Mutable regex lists each time. Regex compilation is expensive and fully redundant.
## Changes
- Memoise compilation in a `sync.Map` keyed by pattern text (`compileCached`); each distinct pattern compiles once and is reused. Patterns that fail to compile are cached as a typed nil so they are not retried. No invalidation needed since the pattern text is the key.
## Validation
- `go test ./internal/proxy/` and `make e2e` pass.
Reviewed-on: #88
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#72
## Why
`compilePatterns` silently discards any pattern that fails to compile. A typo in a blocklist entry therefore turns a deny rule into a no-op — a fail-open with security impact.
## Changes
- Add `Remote.ValidatePatterns`, which compiles every pattern list (patterns, blocklist, mutable/immutable patterns, ban_tags) and returns an error on the first invalid regex.
- Reject invalid patterns with 400 at remote create and update time.
- Unit test for valid and invalid patterns.
## Validation
- `go test ./pkg/models/` and `make e2e` pass.
Reviewed-on: #87
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#68
## Why
`isNetworkError` type-asserted `err.(*UpstreamError)` directly. If the error is ever wrapped, stale-on-error handling silently stops triggering.
## Changes
- Use `errors.As` to detect `*UpstreamError` through wrapping.
## Validation
- `make e2e` passes.
Reviewed-on: #84
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#78
## Why
`serveFromStore` first called `store.Download` with the bare content hash as the S3 key, which never matches real object keys (`blobs/sha256/<hash>`). Every cached blob serve therefore paid an extra guaranteed-404 round-trip before retrying with the correct `BlobKey`.
## Changes
- Remove the dead first `Download` attempt; go straight to the `BlobKey` lookup, then fall back to the index key.
## Validation
- `make e2e` passes (proxy cache-hit paths exercised end-to-end).
Reviewed-on: #82
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Fixes#80
## Why
`make e2e` did not compile against master: `e2e/e2e_test.go` called `server.New(cfg)` but the signature is `New(cfg, version string)`. This blocked all end-to-end validation.
## Changes
- Pass a static `"e2e-test"` version to `server.New` in the e2e bootstrap.
## Validation
- `make e2e` builds and passes (testcontainers: postgres/redis/minio).
Reviewed-on: #81
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Shows total bytes served from cache (instead of upstream) over the last 30 days. Queries `SUM(size_bytes) WHERE cache_hit = TRUE` from access_log.
Reviewed-on: #65
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
When a helm repo base URL includes a path component (e.g. \`stakater.github.io/stakater-charts\`), the merger was extracting the full URL path (\`stakater-charts/reloader-2.2.8.tgz\`) and the proxy then constructed \`base_url/stakater-charts/reloader-2.2.8.tgz\` = double path = 404.
Fix: \`extractPathRelativeToBase()\` strips the shared base path prefix so only the filename portion is used as the proxy path.
Reviewed-on: #64
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
The / endpoint was hardcoded to return 3.0.0-dev. Now uses the git tag version set via ldflags at build time.
Reviewed-on: #63
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Problems
1. Docker daemon sends specific Accept headers to negotiate manifest format, but the proxy dropped them — registries defaulted to OCI format, causing "mediaType should be manifest.v2+json not oci.image.index" errors
2. Upstream Content-Type was only used when the provider returned "application/octet-stream" — Docker manifests got the wrong Content-Type
## Fixes
- Forward client Accept header to upstream (both initial request and Bearer token retry)
- Always prefer upstream Content-Type when present
- Fetch signature now accepts variadic clientHeaders for backwards compat
## E2E tested
- DockerHub: redis:7-alpine, alpine:3 — skopeo inspect OK
- GHCR: OCI-only images work with docker pull (GHCR 404s Docker v2 Accept, which is expected)
- Quay: prometheus/node-exporter — skopeo inspect OK
Reviewed-on: #62
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Vite's \`base: /ui\` makes HTML reference \`/ui/assets/...\` but files are at \`/usr/share/nginx/html/assets/\` (no \`ui/\` subdir). The previous \`location /ui { try_files ... }\` couldn't find the files.
Fix: rewrite strips the base path prefix before try_files, so \`/ui/assets/foo.js\` resolves to \`/usr/share/nginx/html/assets/foo.js\`.
Reviewed-on: #61
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Docker Hub (and other registries) return 401 with a `Www-Authenticate: Bearer realm=...` challenge even for public images. The proxy now:
1. Detects 401 + Bearer challenge
2. Parses realm/service/scope from the header
3. Fetches an anonymous token (or authenticated if username/password configured)
4. Retries the original request with the Bearer token
Fixes: `docker pull artifactapi.../dockerhub/library/redis:latest` returning "unauthorized: upstream returned 401"
Reviewed-on: #60
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
The woodpecker docker-buildx plugin expects build_args as a YAML map (KEY: VALUE), not a list (- KEY=VALUE). The list format was silently ignored, so BASE_PATH was never passed to the Docker build.
Reviewed-on: #59
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Serves the UI under /ui instead of /. This pairs with the argocd route simplification (argocd-apps#201) where /ui → UI service and everything else → API.
- Vite: `base` set from `BASE_PATH` env var at build time
- React Router: `basename` set from injected `__BASE_PATH__`
- Nginx: location block uses `${BASE_PATH}`, substituted by sed at build
- Dockerfile: `ARG BASE_PATH=/` (default preserves existing behavior)
- Woodpecker: passes `BASE_PATH=/ui` to docker-web build
Tested: assets serve at `/ui/assets/...`, SPA routing works at `/ui/remotes`, etc.
Reviewed-on: #58
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
The v3 Go rewrite removed the /v2/ Docker Registry compatibility endpoint. Docker clients need:
- GET/HEAD /v2/ → 200 (registry ping)
- GET/HEAD /v2/{remoteName}/* → proxy to the docker remote
Usage: `docker pull artifactapi.example.com/{remoteName}/image:tag`
Reviewed-on: #57
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Problem
Helm charts like Intel device plugins have download URLs on `github.com` but the chart index is served from `intel.github.io`. The merger rewrites all URLs through the proxy, constructing:
```
https://artifactapi/api/v1/remote/intel-helm/intel/helm-charts/releases/download/...
```
Which proxies to `https://intel.github.io/helm-charts/intel/helm-charts/releases/download/...` — a 404.
## Fix
Compare the download URL host against the remote's base URL host. If they differ, leave the URL as-is so helm downloads directly from the source. Same-host URLs are still rewritten through the proxy.
Also adds `BaseURL` to `MemberIndex` so the merger has the context it needs, and uses the correct `/local/` vs `/remote/` route prefix.
Reviewed-on: #56
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
- New `.pre-commit-config.yaml` with standard Go hooks (gofmt, go vet, go mod tidy) plus file hygiene checks (trailing whitespace, end-of-file, yaml, large files, merge conflicts)
- go vet runs as a local hook with `./...` since the dnephin per-file hook doesn't work with Go module layouts
- Woodpecker pre-commit pipeline updated to use `almalinux9-gobuilder` image with `uvx pre-commit run --all-files`
- Pre-commit hooks installed into the repo
Reviewed-on: #55
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
- New "Locals" sidebar nav item with list + detail + browse pages
- Remotes page filters out local repos (repo_type=local hidden)
- LocalDetail: simplified view — just name, type, description + "Browse Files" button
- Virtuals: member links resolve to /locals/ or /remotes/ based on repo_type
- Objects page detects context for correct back-navigation
## Test plan
- [ ] Visual check: locals page shows only local repos
- [ ] Remotes page hides local repos
- [ ] Virtual member links point to correct pages
- [ ] Browse files works from local detail page
Reviewed-on: #54
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
- Upload RPMs to local repos, metadata parsed async via cavaliergopher/rpm
- Repodata (repomd.xml, primary/filelists/other.xml.gz) generated on-demand from DB — nothing stored in S3
- RPM provider implements LocalUploader, PostUploadHook, and LocalIndexer
- New rpm_metadata table for parsed RPM header data (name, version, deps, etc.)
- New provider interfaces: PostUploadHook, BlobReader, MetadataStore, RPMMetadataReader
## Test plan
- [x] Upload cowsay RPM from epel → async metadata parse confirmed in logs
- [x] repomd.xml generated with correct hashes → primary.xml.gz has correct metadata
- [x] `dnf install` from local repo: download + install successful
- [x] Bad file rejection (.txt → 400), overwrite rejection (409)
Reviewed-on: #53
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
Move package-type-specific local repo logic out of centralized handlers into provider packages via optional Go interfaces.
**New interfaces in `provider` package:**
- \`LocalUploader\`: \`ValidateUpload(filePath) → (storagePath, contentType, error)\` + \`UploadResponse(...)\`
- \`LocalIndexer\`: \`ServeLocalIndex(w, r, files, repoName, path) → bool\` + \`GenerateLocalIndex(ctx, files, repoName, path) → ([]byte, error)\`
- \`FileStore\`: \`ListFilesByPrefix\` + \`ListPackages\` (implemented by database.DB)
**Providers implement these interfaces:**
- PyPI: upload validation (wheel/sdist naming), simple index serving + generation
- Terraform: upload validation (provider zip naming), mirror protocol serving
**Handlers simplified to generic dispatch:**
- \`local.go\`: type-asserts to \`LocalUploader\`, falls back to generic upload
- \`proxy.go\`: type-asserts to \`LocalIndexer\`, falls back to raw file serving
- \`engine.go\`: type-asserts to \`LocalIndexer\` for local virtual members
Adding a new local repo type (e.g. RPM) = implement the interfaces in its provider package. Zero handler changes.
## Test plan
- [x] Build + unit tests pass
- [x] E2E: PyPI local upload → simple index → uv pip install (smoke test after refactor)
Reviewed-on: #52
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
- Virtual engine detects local members and generates indexes in-memory
- MemberIndex.RepoType drives correct URL prefix in merged output
- PyPI merger rewrites links to /api/v1/local/ or /api/v1/remote/ appropriately
- Includes local PyPI support (cherry-picked from #50)
## Test plan
- [x] Upload wheel to local PyPI → install from direct local URL
- [x] Create virtual with local + remote → install from virtual URL
- [x] Both paths produce correct absolute download URLs
Reviewed-on: #51
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
- Upload Python wheels/sdists to local PyPI repos with filename validation
- PEP 503 simple index computed on-demand from stored files
- Package names normalized per PEP 503 (lowercase, hyphens)
- Overwrites rejected (409 Conflict)
## Test plan
- [x] Build wheel with `uv build` → upload → verify simple index HTML → `uv pip install` from local repo
- [x] Bad filename rejection (400)
- [x] Overwrite rejection (409)
- [x] Hash integrity verification on download
Reviewed-on: #50
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Introduces repo_type (remote/local) as a separate axis from package_type
so that any package type can be hosted locally. A terraform local repo
is package_type=terraform + repo_type=local.
- Remote model gains RepoType field (defaults to "remote")
- Database schema adds repo_type column with migration for existing DBs
- V1 proxy adds /api/v1/local/{name}/* route for serving local files
- V2 upload via PUT /api/v2/remotes/{name}/files/{ns}/{type}/{file}.zip
validates filename matches terraform-provider-{type}_{ver}_{os}_{arch}.zip
and returns 409 on duplicate (no overwrites)
- index.json and {version}.json are computed on-the-fly from uploaded zips
rather than stored as separate files
- V2 create validates repo_type and requires base_url only for remotes
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #49
- Objects page renders paths as a collapsible tree instead of flat list
with expand/collapse all, aggregated size/hits per directory
- Dashboard gains top-files-by-hits and top-files-by-bandwidth tables
- Backend: new /api/v2/stats/top-files-by-hits and
/api/v2/stats/top-files-by-bandwidth endpoints
- Raised per_page max to 5000 for objects listing
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #48
## Summary
- New `terraform` package type implementing the [Terraform Registry Protocol](https://developer.hashicorp.com/terraform/internals/provider-registry-protocol)
- `construct_url` prepends `/v1/providers/` so paths like `hashicorp/vault/versions` map to `registry.terraform.io/v1/providers/hashicorp/vault/versions`
- `resolve_content` rewrites `download_url`, `shasums_url`, and `shasums_signature_url` in per-version download info JSON to route through a companion `releases_remote` (generic remote proxying `releases.hashicorp.com`)
- Built-in mutable pattern for `{namespace}/{type}/versions` — version lists expire and are re-fetched; per-version download info is immutable
- Client configuration via `.terraformrc` / `.tofurc` host block — no changes to `.tf` provider source addresses needed
## Test plan
- [x] 8 unit tests covering mutable detection, URL rewriting, binary pass-through, `construct_url` correctness, and cache miss behaviour
- [x] End-to-end: OpenTofu 1.10.3 pulling `hashicorp/vault v4.5.0` through docker-compose stack — `tofu init` succeeded, provider installed and signed
- [x] Verified `download_url` / `shasums_url` rewritten to `hashicorp-releases` proxy in cached response
- [x] All 339 tests pass
Reviewed-on: #45
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Adds two per-remote config keys for docker remotes:
ban_tags_enabled: false # opt-in, default off
ban_tags:
- latest
- edge
When ban_tags_enabled is true and a manifest request arrives for a named
tag in ban_tags, the proxy returns 403. sha256-addressed pulls are never
blocked, so images already pulled can still be referenced by digest.
Blob requests are unaffected.
Reviewed-on: #43
Tag manifests (e.g. library/nginx/manifests/latest) and their sha256-addressed
counterparts were stored at separate S3 keys with no cross-reference, so a
sha256 manifest request always missed cache even when the identical content had
just been stored under the tag key.
After serving any mutable (tag) manifest, compute the sha256 of the response
body and write it under the digest key (manifests/sha256:<hex>) if absent. The
next sha256-addressed pull hits cache immediately.
Also adds a short-lived Redis distributed lock (SET NX EX 30) around upstream
fetches so that concurrent pods racing for the same cold key poll storage for
up to 5 s before issuing a duplicate upstream request, eliminating the
thundering herd on deploy events.
Includes unit tests for both the lock primitives (acquire/release, fail-open
when Redis is unavailable) and the docker proxy behaviour (cross-link written
on tag hit, not written for sha256 requests, lock acquired/released, poll path
serves from cache without upstream fetch, fallback fetch when poll times out).
Reviewed-on: #42
Closes#34
## Summary
- At module load time, a `try/except` selects `yaml.CSafeLoader` / `yaml.CDumper` (C extensions) when libyaml is available, otherwise falls back to `yaml.SafeLoader` / `yaml.Dumper`
- `_HelmDumper` inherits from whichever dumper base was selected — custom datetime/date representers are registered the same way as before
- `_merge_helm_indexes` uses `yaml.load(raw_data, Loader=_YamlLoader)` instead of `yaml.safe_load`
- No change to `yaml.dump(...)` call — it already passes `Dumper=_HelmDumper`, which now inherits from the C base when available
- Five new tests in `TestYamlExtensionSelection` cover: loader/dumper base are classes, `_HelmDumper` inherits from the selected base, C extensions used when available, loader can parse YAML
## Measured performance gain
19-member `helm-all` virtual repo, real upstream data, Docker (AlmaLinux 9):
| | `merge=` time |
|---|---|
| Before (SafeLoader + Dumper) | **38,877ms** |
| After (CSafeLoader + CDumper) | **9,625ms** |
| Speedup | **4.0×** |
Local microbenchmark (500 charts × 10 versions × 19 members, 3 runs avg):
- Before: **40.8s** → After: **6.1s** (**6.7×** faster)
## Test plan
- [x] 283 unit tests pass (`make test`)
- [x] Wheel builds cleanly (`uv build --wheel`)
- [x] C extension confirmed available in AlmaLinux 9 container: `yaml.CSafeLoader: <class 'yaml.cyaml.CSafeLoader'>`
- [x] Baseline Docker timing measured with pure-Python path forced: merge=38,877ms
- [x] After Docker timing measured with C extension path: merge=9,625ms
Reviewed-on: #39
Closes#35
## Summary
- Wraps `handler.merge(...)` in `await asyncio.to_thread(...)` so the CPU-bound YAML parse/merge/dump runs in the thread pool instead of blocking the event loop
- Change is at the generic `handle()` dispatch site — applies to all current and future `_VirtualHandler` implementations without modification
- Also fixes a pre-existing bug in `examples/single-file/remotes.yaml` where `base_url` and `package` keys were merged onto a single line, preventing `docker-compose up` from starting the app
## Measured performance gain
19-member `helm-all` virtual repo, single uvicorn worker, cache miss (38s merge):
| | Concurrent `/health` latency |
|---|---|
| Before (blocking) | **37,721ms** for first request (stalled) |
| After (thread pool) | **8–63ms** for all requests |
## Test plan
- [x] 278 unit tests pass (`make test`)
- [x] Live concurrency test: cache miss merge started in background, 5 concurrent `/health` checks measured — all <65ms
- [x] Baseline comparison: same test with blocking call — first health check stalled 37.7s
Reviewed-on: #38
Closes#33
## Summary
- `_merge_helm_indexes` now parses each member's raw YAML first, then rewrites `urls` entries in-place via the new `_rewrite_urls` helper
- **Relative URLs** (e.g. `rancher-2.13.1.tgz`) are prepended with `{proxy_base}/api/v1/remote/{member_name}/`
- **Absolute URLs** matching `base_url` are rewritten to the proxy path (existing behaviour, now correct)
- **Absolute URLs** with a different prefix are left unchanged
- Removes the `_helm.resolve_content` raw-bytes detour from the virtual merge path; `remote/helm.py` is unchanged (still used for direct remote proxying)
## Test plan
- [x] 278 unit tests pass (`make test`)
- [x] New `TestRewriteUrls` class covering relative, absolute-match, absolute-no-match, leading-slash, and multi-URL cases
- [x] New `test_relative_urls_rewritten_to_proxy` in `TestMergeHelmIndexes`
- [x] Updated `test_first_member_wins_on_duplicate_name_and_version` to assert on proxy remote name (not upstream hostname)
- [x] Live Docker test: Rancher `index.yaml` relative URLs rewritten correctly to `http://localhost:8000/api/v1/remote/rancher-stable/rancher-2.14.1.tgz` etc.
- [x] `helm-all` virtual (19 members) returns HTTP 200 with 395k-line merged index on cache miss
Reviewed-on: #37
Repository types now live under dedicated top-level keys instead of a
shared remotes: block distinguished by a type field:
remotes: caching proxy remotes (no type field needed)
virtuals: virtual merged-index repositories
locals: local upload repositories
Routes for local repos move from /api/v1/remote/ to /api/v1/local/.
config.py gains get_virtual_config() and get_local_config() lookups.
Root endpoint now reports all three sections. Drop root conf.d/ (was
an exact duplicate of examples/conf.d-method/).
Reviewed-on: #31
Adds a new virtual repo type that merges indexes from multiple member remotes
of the same package type. Currently supports helm (index.yaml merge with URL
rewriting). Member fetches run in parallel; merged index is Redis-cached at
min(mutable_ttl) across members.
Reviewed-on: #30
examples/single-file/remotes.yaml — original monolithic config
examples/conf.d-method/ — one yaml per remote (alpine, github, pypi)
docker-compose updated to mount from examples/single-file/.
CONFIG_PATH now accepts a directory path (all *.yaml files merged) or a
main file with a config_dir key pointing to a drop-in directory. Remotes
are merged alphabetically across files; later files win on conflicts.
unkinben
benvin