Document sync_interval for deb/alpine and fix created_at gate on existing rows (#176)
`sync_interval` is documented only for github_rpm, and the claim gate holding back remotes created inside the current slot also blocks remotes that already have a sync state row: the gate filters the INSERT SELECT, so ON CONFLICT never runs and a failed cold-cache prime waits a full slot instead of its backoff. - document `sync_interval` for github_deb/github_alpine and in the caching field table - apply the created_at gate only when no sync state row exists - test 0→N and N→0 interval changes and the insert-only gate against Postgres - test that a negative `sync_interval` on update returns 400 Reviewed-on: #176 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net>
This commit was merged in pull request #176.
This commit is contained in:
@@ -5,3 +5,16 @@ Shared classification, TTL and eviction rules: [caching](../common/caching.md).
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `github_alpine` | `APKINDEX.tar.gz` (synthesized) | `.apk` (redirected) |
|
||||
|
||||
## Release sync
|
||||
|
||||
A background syncer re-scans the GitHub releases list per remote, sharing one
|
||||
lease per remote across replicas through `github_alpine_sync_state`.
|
||||
|
||||
- **`sync_interval`** (seconds) re-checks once per interval at a stable
|
||||
per-remote offset, so remotes on the same interval spread out instead of
|
||||
syncing together; its first scheduled sync waits for that slot (a cold-cache
|
||||
request still primes at once). Unset, it re-checks no more often than its
|
||||
`mutable_ttl`.
|
||||
- A failed scan keeps serving the last good index and retries with exponential
|
||||
backoff.
|
||||
|
||||
@@ -13,6 +13,7 @@ Every proxied request on a remote is classified as **denied**, **immutable** or
|
||||
| `mutable_patterns` | Regexes forcing a path mutable. |
|
||||
| `immutable_ttl` | Seconds an immutable path stays fresh. `0` = forever. |
|
||||
| `mutable_ttl` | Seconds a mutable path stays fresh. `0` also means never expires. Omitted on create, it defaults to `3600`; omitted on update, the stored value is kept. |
|
||||
| `sync_interval` | github_rpm/github_deb/github_alpine only. Seconds between background release re-scans, at a stable per-remote offset. Unset (`0`) re-scans no more often than `mutable_ttl`. |
|
||||
| `check_mutable` | When a mutable path's `mutable_ttl` expires, sends a conditional `HEAD` (`If-None-Match`) with the stored upstream ETag. A `304` re-arms the TTL and serves the stored copy; anything else refetches in full. The ETag is kept for `max(10 × mutable_ttl, 7 days)`, re-armed on each `304`, so idle paths age out. It is only stored when `check_mutable` is set and `mutable_ttl` is non-zero, and never for immutable paths. |
|
||||
| `stale_on_error` | On an upstream network error, serve the stored copy instead of failing. |
|
||||
| `serve_stale` | rpm/deb/alpine virtual members only. When a virtual re-merges, use this member's previous index at once and refresh it in the background; the next merge picks up the refresh. The first merge still waits, and so does any merge once the previous index is over an hour old, so a member that keeps failing surfaces its error. Default `false`. |
|
||||
|
||||
@@ -5,3 +5,16 @@ Shared classification, TTL and eviction rules: [caching](../common/caching.md).
|
||||
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|
||||
|---|---|---|
|
||||
| `github_deb` | `Packages*`, `Release*` (synthesized) | `.deb` (redirected) |
|
||||
|
||||
## Release sync
|
||||
|
||||
A background syncer re-scans the GitHub releases list per remote, sharing one
|
||||
lease per remote across replicas through `github_deb_sync_state`.
|
||||
|
||||
- **`sync_interval`** (seconds) re-checks once per interval at a stable
|
||||
per-remote offset, so remotes on the same interval spread out instead of
|
||||
syncing together; its first scheduled sync waits for that slot (a cold-cache
|
||||
request still primes at once). Unset, it re-checks no more often than its
|
||||
`mutable_ttl`.
|
||||
- A failed scan keeps serving the last good index and retries with exponential
|
||||
backoff.
|
||||
|
||||
@@ -37,9 +37,9 @@ derived metadata current off the client request path:
|
||||
stable per-remote offset, so remotes on the same interval spread out instead
|
||||
of syncing together; its first scheduled sync waits for that slot (a
|
||||
cold-cache request still primes at once). Unset, it re-checks no more often
|
||||
than its `mutable_ttl`. New/changed
|
||||
assets are derived incrementally; assets already cached are never re-fetched,
|
||||
and assets that disappear upstream are pruned.
|
||||
than its `mutable_ttl`. New/changed assets are derived incrementally; assets
|
||||
already cached are never re-fetched, and assets that disappear upstream are
|
||||
pruned.
|
||||
- **ETag / 304 conditional requests.** The releases-list `ETag` is stored per
|
||||
remote and sent as `If-None-Match`; a `304 Not Modified` means nothing changed
|
||||
and the syncer derives nothing. GitHub does not count `304` conditional
|
||||
@@ -54,8 +54,8 @@ derived metadata current off the client request path:
|
||||
periodic scan a replica must atomically claim a per-remote lease
|
||||
(`github_rpm_sync_state`: `last_synced_at`, `etag`, `sync_lease_owner`,
|
||||
`sync_lease_expires`); only the winner scans. This bounds total GitHub load to
|
||||
~once per `sync_interval` (else `mutable_ttl`) regardless of replica count, and the shared `etag`
|
||||
lets any replica issue the conditional request.
|
||||
~once per `sync_interval` (else `mutable_ttl`) regardless of replica count,
|
||||
and the shared `etag` lets any replica issue the conditional request.
|
||||
|
||||
```hcl
|
||||
# Backend that serves the actual .rpm bytes from github.com.
|
||||
|
||||
Reference in New Issue
Block a user