unkin/artifactapi
2
0
Fork 0
Code Issues 5 Pull Requests 2 Actions Packages Projects Releases 1 Wiki Activity

fix: coalesce concurrent cache-miss fetches (thundering herd) #93

Merged
benvin merged 1 commits from benvin/coalesce-cache-miss into master 2026-07-02 22:08:30 +10:00
Conversation 0 Commits 1 Files Changed 1
+29 -3
unkinben commented 2026-07-02 00:44:09 +10:00
Owner
Copy Link
Copy Source

Fixes #75

Why

On a fetch-lock miss, Engine.Fetch slept a flat 500ms once, tried the store, and otherwise fell through to fetch upstream unlocked. A cold-cache stampede therefore still hit upstream once per waiter.

Changes

  • Add waitForStore, which polls the store every 100ms for up to 5s (stopping on context cancellation) so waiters pick up the lock leaders populated result.
  • Only fall through to an upstream fetch if the leader has not populated the store within the wait budget.

Validation

  • make e2e passes.
Fixes #75 ## Why On a fetch-lock miss, `Engine.Fetch` slept a flat 500ms once, tried the store, and otherwise fell through to fetch upstream unlocked. A cold-cache stampede therefore still hit upstream once per waiter. ## Changes - Add `waitForStore`, which polls the store every 100ms for up to 5s (stopping on context cancellation) so waiters pick up the lock leaders populated result. - Only fall through to an upstream fetch if the leader has not populated the store within the wait budget. ## Validation - `make e2e` passes.
unkinben added 1 commit 2026-07-02 00:44:09 +10:00
fix: poll the store on lock miss to coalesce cache-miss stampedes
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/build Pipeline was successful
Details
a3b972e7b1 
On a fetch-lock miss the engine slept a flat 500ms once and, on a store
miss, fell through to fetch upstream unlocked anyway, so a cold-cache
stampede hit upstream once per waiter. Poll the store for up to 5s (100ms
interval, context-aware) so waiters pick up the leader's result instead of
duplicating the upstream fetch.

Refs #75
benvin merged commit 5261af4c63 into master 2026-07-02 22:08:30 +10:00
benvin deleted branch benvin/coalesce-cache-miss 2026-07-02 22:08:30 +10:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/artifactapi#93
Delete Branch "benvin/coalesce-cache-miss"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?