perf: offload virtual repo merge to thread pool via asyncio.to_thread #38

2026-05-02T01:30:51+10:00

unkinben commented

2026-05-02 01:30:51 +10:00

Closes #35

Summary

Wraps handler.merge(...) in await asyncio.to_thread(...) so the CPU-bound YAML parse/merge/dump runs in the thread pool instead of blocking the event loop
Change is at the generic handle() dispatch site — applies to all current and future _VirtualHandler implementations without modification
Also fixes a pre-existing bug in examples/single-file/remotes.yaml where base_url and package keys were merged onto a single line, preventing docker-compose up from starting the app

Measured performance gain

19-member helm-all virtual repo, single uvicorn worker, cache miss (38s merge):

	Concurrent `/health` latency
Before (blocking)	37,721ms for first request (stalled)
After (thread pool)	8–63ms for all requests

Test plan

278 unit tests pass (make test)
Live concurrency test: cache miss merge started in background, 5 concurrent /health checks measured — all <65ms
Baseline comparison: same test with blocking call — first health check stalled 37.7s

Closes #35 ## Summary - Wraps `handler.merge(...)` in `await asyncio.to_thread(...)` so the CPU-bound YAML parse/merge/dump runs in the thread pool instead of blocking the event loop - Change is at the generic `handle()` dispatch site — applies to all current and future `_VirtualHandler` implementations without modification - Also fixes a pre-existing bug in `examples/single-file/remotes.yaml` where `base_url` and `package` keys were merged onto a single line, preventing `docker-compose up` from starting the app ## Measured performance gain 19-member `helm-all` virtual repo, single uvicorn worker, cache miss (38s merge): | | Concurrent `/health` latency | |---|---| | Before (blocking) | **37,721ms** for first request (stalled) | | After (thread pool) | **8–63ms** for all requests | ## Test plan - [x] 278 unit tests pass (`make test`) - [x] Live concurrency test: cache miss merge started in background, 5 concurrent `/health` checks measured — all <65ms - [x] Baseline comparison: same test with blocking call — first health check stalled 37.7s

unkinben added 1 commit 2026-05-02 01:30:52 +10:00

perf: offload virtual repo merge to thread pool via asyncio.to_thread

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

6e5aa578ee

The synchronous handler.merge() call blocked the uvicorn event loop for
the entire merge duration (38s for a 19-member helm virtual repo), stalling
all other requests on the same worker during that window.

Wrapping the call in asyncio.to_thread() releases the event loop while the
CPU-bound YAML parse/merge/dump runs in the default thread pool, allowing
health checks and other requests to be served concurrently.

Measured impact (19-member helm-all, single worker):
  Before: first concurrent /health stalled for 37721ms
  After:  all concurrent /health requests served in <65ms

The change is at the generic handler dispatch site, so it applies to all
current and future _VirtualHandler implementations without modification.

Also fix base_url/package keys merged onto a single line in
examples/single-file/remotes.yaml, which prevented docker-compose startup.

unkinben merged commit 7b6c69b70f into master

2026-05-02 01:35:45 +10:00

unkinben deleted branch benvin/issue-35-thread-pool-merge

2026-05-02 01:35:46 +10:00

unkinben referenced this issue from a commit

2026-05-02 01:35:47 +10:00

perf: offload virtual repo merge to thread pool via asyncio.to_thread (#38)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/artifactapi#38