feat: cache parsed member indexes as msgpack to skip YAML re-parse on rebuild
Warm rebuilds of virtual repos (member caches valid, virtual TTL expired) previously re-parsed all member index.yaml files on every rebuild. With 19 Helm members totalling 14 MB, YAML parsing was 60% of merge time (~6.3s of ~9.6s). Parsing each member's YAML also produces msgpack and stores it in S3 alongside the raw index. Subsequent rebuilds load the compact msgpack and skip YAML parsing entirely. Before: warm rebuild ~9.6s (CSafeLoader baseline) After: warm rebuild ~5.9s (38% faster, merge=4.7s down from ~9.6s)
This commit is contained in:
@@ -14,6 +14,7 @@ dependencies = [
|
||||
"lxml>=4.9.0",
|
||||
"prometheus-client>=0.19.0",
|
||||
"python-multipart>=0.0.6",
|
||||
"msgpack>=1.0.0",
|
||||
]
|
||||
requires-python = ">=3.11"
|
||||
readme = "README.md"
|
||||
|
||||
Reference in New Issue
Block a user