unkin/artifactapi
2
0
Fork 0
Code Issues 5 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

feat: cache parsed member indexes as msgpack to skip YAML re-parse on rebuild #40

Merged
unkinben merged 1 commits from benvin/issue-36-msgpack-member-cache into master 2026-05-02 17:15:31 +10:00
Conversation 0 Commits 1 Files Changed 4
+284 -41

1 Commits

Author SHA1 Message Date
unkinben 1e0f4dc840 feat: cache parsed member indexes as msgpack to skip YAML re-parse on rebuild
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/build Pipeline was successful
Details 
Warm rebuilds of virtual repos (member caches valid, virtual TTL expired)
previously re-parsed all member index.yaml files on every rebuild. With 19
Helm members totalling 14 MB, YAML parsing was 60% of merge time (~6.3s of
~9.6s). Parsing each member's YAML also produces msgpack and stores it in S3
alongside the raw index. Subsequent rebuilds load the compact msgpack and skip
YAML parsing entirely.

Before: warm rebuild ~9.6s (CSafeLoader baseline)
After:  warm rebuild ~5.9s (38% faster, merge=4.7s down from ~9.6s)
 2026-05-02 17:04:19 +10:00