Fix mutable path refresh: default mutable_ttl, revalidate after expiry #151

Merged

benvin merged 3 commits from benvin/mutable-refresh into master 2026-10-17 17:17:41 +11:00

Conversation 2 Commits 3 Files Changed 9

+286 -37

unkin-agent commented 2026-10-17 16:25:40 +11:00

Member

Copy Link

Copy Source

A remote created through the API without mutable_ttl was stored as 0, so its mutable paths never refreshed. With check_mutable, the ETag expired together with the freshness key, so an expired mutable path was always refetched in full and never revalidated.

• default an omitted mutable_ttl to 3600 on create; keep the stored value on update; explicit 0 still means never expire
• keep mutable ETags for max(10 × mutable_ttl, 7d), re-armed on each 304, only when check_mutable is set and mutable_ttl is non-zero
• drop the stored ETag when a refetch returns none; never store one for immutable paths
• flush a remote's Redis keys on delete
• update docs/common/caching.md

A remote created through the API without `mutable_ttl` was stored as 0, so its mutable paths never refreshed. With `check_mutable`, the ETag expired together with the freshness key, so an expired mutable path was always refetched in full and never revalidated. - default an omitted `mutable_ttl` to 3600 on create; keep the stored value on update; explicit 0 still means never expire - keep mutable ETags for `max(10 × mutable_ttl, 7d)`, re-armed on each 304, only when `check_mutable` is set and `mutable_ttl` is non-zero - drop the stored ETag when a refetch returns none; never store one for immutable paths - flush a remote's Redis keys on delete - update docs/common/caching.md

unkin-agent added 1 commit 2026-10-17 16:25:41 +11:00

Default omitted mutable_ttl and keep the ETag past TTL expiry ... 9eb8ef384f

- default mutable_ttl to 3600 on create and keep the stored value on update
- store mutable ETags without a TTL so expiry revalidates with If-None-Match
- drop the stored ETag when a refetch returns none

unkin-agent commented 2026-10-17 16:40:12 +11:00

Author

Member

Copy Link

Copy Source

• internal/proxy/engine.go:494 — SetETag(..., 0) writes a never-expiring etag:<remote>:<path> key for every mutable path of every remote, including check_mutable=false remotes that never read it; nothing expires or GCs it, so keys for paths never requested again leak forever → write it only when remote.CheckMutable, with a bounded TTL (e.g. a multiple of mutable_ttl or a fixed 7-30d cap) instead of 0.
• internal/api/v2/remotes.go:167 — del never clears Redis, so the now-permanent etag:* keys survive remote deletion (and a same-name re-create can 304 against a stale ETag) → call h.cache.FlushRemote(ctx, name) after DeleteRemote, with a test.
• internal/proxy/engine_test.go:263 — no test for a 304 whose body was evicted from S3, stale_on_error with a failing HEAD, or etag cleanup on remote delete → add them.
• nit: internal/proxy/engine_test.go:285 — two real time.Sleep(1100ms) waits slow the suite and can flake under -race → expire the freshness key via testCache.SetTTL(..., time.Millisecond).

- internal/proxy/engine.go:494 — `SetETag(..., 0)` writes a never-expiring `etag:<remote>:<path>` key for every mutable path of every remote, including `check_mutable=false` remotes that never read it; nothing expires or GCs it, so keys for paths never requested again leak forever → write it only when `remote.CheckMutable`, with a bounded TTL (e.g. a multiple of `mutable_ttl` or a fixed 7-30d cap) instead of 0. - internal/api/v2/remotes.go:167 — `del` never clears Redis, so the now-permanent `etag:*` keys survive remote deletion (and a same-name re-create can 304 against a stale ETag) → call `h.cache.FlushRemote(ctx, name)` after `DeleteRemote`, with a test. - internal/proxy/engine_test.go:263 — no test for a 304 whose body was evicted from S3, `stale_on_error` with a failing HEAD, or etag cleanup on remote delete → add them. - nit: internal/proxy/engine_test.go:285 — two real `time.Sleep(1100ms)` waits slow the suite and can flake under `-race` → expire the freshness key via `testCache.SetTTL(..., time.Millisecond)`.

unkin-agent added 1 commit 2026-10-17 16:51:39 +11:00

Bound the mutable ETag TTL and flush Redis state on remote delete 3c76902533

unkin-agent added 1 commit 2026-10-17 16:57:00 +11:00

Stop storing ETags for immutable paths 6a7eff9379

unkin-agent commented 2026-10-17 17:01:06 +11:00

Author

Member

Copy Link

Copy Source

No findings.

No findings.

benvin merged commit d69a58e54a into master 2026-10-17 17:17:41 +11:00

benvin deleted branch benvin/mutable-refresh 2026-10-17 17:17:42 +11:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/artifactapi#151