feat: cache parsed member indexes as msgpack to skip YAML re-parse on rebuild
Warm rebuilds of virtual repos (member caches valid, virtual TTL expired) previously re-parsed all member index.yaml files on every rebuild. With 19 Helm members totalling 14 MB, YAML parsing was 60% of merge time (~6.3s of ~9.6s). Parsing each member's YAML also produces msgpack and stores it in S3 alongside the raw index. Subsequent rebuilds load the compact msgpack and skip YAML parsing entirely. Before: warm rebuild ~9.6s (CSafeLoader baseline) After: warm rebuild ~5.9s (38% faster, merge=4.7s down from ~9.6s)
This commit is contained in:
@@ -390,7 +390,7 @@ If a member is unreachable and has no cached index, it is skipped and a warning
|
||||
|
||||
**Caching:**
|
||||
|
||||
The merged index is cached using `min(mutable_ttl)` across all members. Each member's raw index is cached in S3 under its own remote key by the normal proxy rules; the virtual handler reuses those copies when available.
|
||||
The merged index is cached using `min(mutable_ttl)` across all members. Each member's raw index is cached in S3 under its own remote key; the virtual handler reuses those copies when available. On rebuild, each member's parsed index is also stored as a compact msgpack file (`index.msgpack`) alongside the raw YAML, eliminating the YAML parse cost on subsequent rebuilds.
|
||||
|
||||
**Helm example:**
|
||||
|
||||
|
||||
Reference in New Issue
Block a user