unkin/artifactapi
2
0
Fork 0
Code Issues 5 Pull Requests 2 Actions Packages Projects Releases 1 Wiki Activity

perf: batch access-log writes instead of goroutine+insert per request #91

Merged
benvin merged 1 commits from benvin/batch-access-log into master 2026-07-02 22:07:56 +10:00
Conversation 0 Commits 1 Files Changed 2
+100 -17
unkinben commented 2026-07-02 00:40:46 +10:00
Owner
Copy Link
Copy Source

Fixes #76

Why

Every proxied request spawned a goroutine running a 5s-timeout single-row INSERT. Under load this is unbounded goroutines and connection-pool pressure.

Changes

  • Add database.AccessLogEntry + InsertAccessLogBatch (bulk COPY).
  • The engine starts one background writer that drains a buffered channel and flushes every 128 entries or 2s.
  • logAccess is now a non-blocking channel send (drops on full buffer), so the request path never blocks on the DB. Best-effort telemetry: a small tail may be lost on abrupt shutdown.

Validation

  • make e2e passes.
Fixes #76 ## Why Every proxied request spawned a goroutine running a 5s-timeout single-row INSERT. Under load this is unbounded goroutines and connection-pool pressure. ## Changes - Add `database.AccessLogEntry` + `InsertAccessLogBatch` (bulk `COPY`). - The engine starts one background writer that drains a buffered channel and flushes every 128 entries or 2s. - `logAccess` is now a non-blocking channel send (drops on full buffer), so the request path never blocks on the DB. Best-effort telemetry: a small tail may be lost on abrupt shutdown. ## Validation - `make e2e` passes.
unkinben added 1 commit 2026-07-02 00:40:46 +10:00
perf: batch access-log writes via a buffered background writer
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/build Pipeline was successful
Details
41f069bf96 
Every request spawned a goroutine doing a 5s-timeout INSERT, unbounded
under load. Route entries through a buffered channel drained by a single
background writer that bulk-inserts with COPY every 128 entries or 2s.
logAccess is now a non-blocking send that drops on a full buffer, so the
request path never waits on the database.

Refs #76
benvin merged commit 45d6cdbc64 into master 2026-07-02 22:07:56 +10:00
benvin deleted branch benvin/batch-access-log 2026-07-02 22:07:56 +10:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/artifactapi#91
Delete Branch "benvin/batch-access-log"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?