Document sync_interval for deb/alpine and fix created_at gate on existing rows (#176)

`sync_interval` is documented only for github_rpm, and the claim gate holding back remotes created inside the current slot also blocks remotes that already have a sync state row: the gate filters the INSERT SELECT, so ON CONFLICT never runs and a failed cold-cache prime waits a full slot instead of its backoff.

- document `sync_interval` for github_deb/github_alpine and in the caching field table
- apply the created_at gate only when no sync state row exists
- test 0→N and N→0 interval changes and the insert-only gate against Postgres
- test that a negative `sync_interval` on update returns 400

Reviewed-on: #176
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
This commit was merged in pull request #176.
This commit is contained in:
2026-10-25 20:26:34 +11:00
committed by BenVincent
parent 00edeb478b
commit 7f65fc377c
7 changed files with 129 additions and 7 deletions
+13
View File
@@ -5,3 +5,16 @@ Shared classification, TTL and eviction rules: [caching](../common/caching.md).
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|---|---|---|
| `github_alpine` | `APKINDEX.tar.gz` (synthesized) | `.apk` (redirected) |
## Release sync
A background syncer re-scans the GitHub releases list per remote, sharing one
lease per remote across replicas through `github_alpine_sync_state`.
- **`sync_interval`** (seconds) re-checks once per interval at a stable
per-remote offset, so remotes on the same interval spread out instead of
syncing together; its first scheduled sync waits for that slot (a cold-cache
request still primes at once). Unset, it re-checks no more often than its
`mutable_ttl`.
- A failed scan keeps serving the last good index and retries with exponential
backoff.
+1
View File
@@ -13,6 +13,7 @@ Every proxied request on a remote is classified as **denied**, **immutable** or
| `mutable_patterns` | Regexes forcing a path mutable. |
| `immutable_ttl` | Seconds an immutable path stays fresh. `0` = forever. |
| `mutable_ttl` | Seconds a mutable path stays fresh. `0` also means never expires. Omitted on create, it defaults to `3600`; omitted on update, the stored value is kept. |
| `sync_interval` | github_rpm/github_deb/github_alpine only. Seconds between background release re-scans, at a stable per-remote offset. Unset (`0`) re-scans no more often than `mutable_ttl`. |
| `check_mutable` | When a mutable path's `mutable_ttl` expires, sends a conditional `HEAD` (`If-None-Match`) with the stored upstream ETag. A `304` re-arms the TTL and serves the stored copy; anything else refetches in full. The ETag is kept for `max(10 × mutable_ttl, 7 days)`, re-armed on each `304`, so idle paths age out. It is only stored when `check_mutable` is set and `mutable_ttl` is non-zero, and never for immutable paths. |
| `stale_on_error` | On an upstream network error, serve the stored copy instead of failing. |
| `serve_stale` | rpm/deb/alpine virtual members only. When a virtual re-merges, use this member's previous index at once and refresh it in the background; the next merge picks up the refresh. The first merge still waits, and so does any merge once the previous index is over an hour old, so a member that keeps failing surfaces its error. Default `false`. |
+13
View File
@@ -5,3 +5,16 @@ Shared classification, TTL and eviction rules: [caching](../common/caching.md).
| Type | Mutable (auto-detected) | Immutable (auto-detected) |
|---|---|---|
| `github_deb` | `Packages*`, `Release*` (synthesized) | `.deb` (redirected) |
## Release sync
A background syncer re-scans the GitHub releases list per remote, sharing one
lease per remote across replicas through `github_deb_sync_state`.
- **`sync_interval`** (seconds) re-checks once per interval at a stable
per-remote offset, so remotes on the same interval spread out instead of
syncing together; its first scheduled sync waits for that slot (a cold-cache
request still primes at once). Unset, it re-checks no more often than its
`mutable_ttl`.
- A failed scan keeps serving the last good index and retries with exponential
backoff.
+5 -5
View File
@@ -37,9 +37,9 @@ derived metadata current off the client request path:
stable per-remote offset, so remotes on the same interval spread out instead
of syncing together; its first scheduled sync waits for that slot (a
cold-cache request still primes at once). Unset, it re-checks no more often
than its `mutable_ttl`. New/changed
assets are derived incrementally; assets already cached are never re-fetched,
and assets that disappear upstream are pruned.
than its `mutable_ttl`. New/changed assets are derived incrementally; assets
already cached are never re-fetched, and assets that disappear upstream are
pruned.
- **ETag / 304 conditional requests.** The releases-list `ETag` is stored per
remote and sent as `If-None-Match`; a `304 Not Modified` means nothing changed
and the syncer derives nothing. GitHub does not count `304` conditional
@@ -54,8 +54,8 @@ derived metadata current off the client request path:
periodic scan a replica must atomically claim a per-remote lease
(`github_rpm_sync_state`: `last_synced_at`, `etag`, `sync_lease_owner`,
`sync_lease_expires`); only the winner scans. This bounds total GitHub load to
~once per `sync_interval` (else `mutable_ttl`) regardless of replica count, and the shared `etag`
lets any replica issue the conditional request.
~once per `sync_interval` (else `mutable_ttl`) regardless of replica count,
and the shared `etag` lets any replica issue the conditional request.
```hcl
# Backend that serves the actual .rpm bytes from github.com.